@zosmaai/pi-llm-wiki 0.11.7 → 0.12.0

package/docs/superpowers/plans/2026-08-29-custom-wiki-page-types.md

@@ -0,0 +1,378 @@
+# Custom Wiki Page Types Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Allow users to register custom page types via settings, so `wiki_ensure_page` supports project-specific taxonomies alongside the 7 built-in types.
+
+**Architecture:** Add `customTypes` to `TaskConfig` (existing settings surface). At runtime, merge custom types into `wiki_ensure_page`'s folder map. Custom types get a generic page template (no per-type templates — YAGNI). Works transparently for both pi and oh-my-pi via the existing `settings.json` / `config.yml` pipeline.
+
+**Tech Stack:** TypeScript, Vitest
+
+**Roadmap:** None
+
+**Phase:** Single-plan implementation
+
+---
+
+## File Map
+
+| File | Change |
+|------|--------|
+| `extensions/llm-wiki/lib/task-config.ts:33` | Add `customTypes` field to `TaskConfig` interface |
+| `extensions/llm-wiki/lib/task-config.ts:225` | Add parsing block in `readNamespacedConfig` |
+| `extensions/llm-wiki/lib/task-config.ts:488` | Add `"customTypes"` to `KNOWN_KEYS` array |
+| `extensions/llm-wiki/lib/tools.ts:520` | Update type parameter description |
+| `extensions/llm-wiki/lib/tools.ts:540` | Merge config custom types into `folderMap` |
+| `test/custom-types.test.ts` | New test file for custom type behavior |
+
+---
+
+## Task 1: Add `customTypes` to TaskConfig
+
+**Files:**
+- Modify: `extensions/llm-wiki/lib/task-config.ts:33-165` (interface)
+- Modify: `extensions/llm-wiki/lib/task-config.ts:225-320` (readNamespacedConfig)
+- Modify: `extensions/llm-wiki/lib/task-config.ts:488` (KNOWN_KEYS)
+
+- [ ] **Step 1: Add the field to TaskConfig interface**
+
+Insert after the `wikilinkValidation` field (line ~165):
+
+```ts
+  /**
+   * User-defined page types for wiki_ensure_page (issue #169). Merges with
+   * the 7 built-in types (entity, concept, synthesis, analysis, requirement,
+   * skill, case). Each key is the type name; each value is the folder name
+   * inside wiki/. Example: { "decision": "decisions", "metric": "metrics" }
+   * → wiki_ensure_page(type="decision") creates wiki/decisions/<slug>.md.
+   * Custom types use a generic page template (no per-type templates).
+   */
+  customTypes?: Record<string, string>;
+```
+
+- [ ] **Step 2: Add parsing in `readNamespacedConfig`**
+
+Insert before the `return out;` line (line ~318), after the `wikilinkValidation` block:
+
+```ts
+    const ct = section.customTypes;
+    if (ct && typeof ct === "object" && !Array.isArray(ct)) {
+      const entries = Object.entries(ct as Record<string, unknown>);
+      const valid: Record<string, string> = {};
+      for (const [k, v] of entries) {
+        if (typeof k === "string" && typeof v === "string" && k && v) {
+          valid[k] = v;
+        }
+      }
+      if (Object.keys(valid).length) out.customTypes = valid;
+    }
+```
+
+This validates that customTypes is a flat `Record<string, string>` — rejects arrays, nested objects, and non-string values.
+
+- [ ] **Step 3: Add to KNOWN_KEYS**
+
+In the `KNOWN_KEYS` array (line ~488), add `"customTypes"` after `"wikilinkValidation"`:
+
+```ts
+const KNOWN_KEYS = [
+  "taskModel",
+  "embeddingProvider",
+  "embeddingModel",
+  "embeddingBaseUrl",
+  "embeddingApiKey",
+  "embeddingApiKeyEnv",
+  "semanticWeight",
+  "recallLinksThreshold",
+  "recallSkillInlineMax",
+  "notices",
+  "ambientPersonalVault",
+  "trajectories",
+  "synthesisLanguage",
+  "synthesisMaxTokens",
+  "wikilinkValidation",
+  "customTypes",
+] as const;
+```
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add extensions/llm-wiki/lib/task-config.ts
+git commit -m "feat(config): add customTypes to TaskConfig with readNamespacedConfig parsing"
+```
+
+---
+
+## Task 2: Merge custom types into wiki_ensure_page
+
+**Files:**
+- Modify: `extensions/llm-wiki/lib/tools.ts:520` (type description)
+- Modify: `extensions/llm-wiki/lib/tools.ts:540-564` (folderMap + type cast)
+
+- [ ] **Step 1: Load config and merge folderMap**
+
+Replace the hardcoded `folderMap` and type cast (lines ~540-564) with:
+
+```ts
+      const config = loadTaskConfig(ctx.cwd);
+      const builtInFolderMap: Record<string, string> = {
+        entity: "entities",
+        concept: "concepts",
+        synthesis: "syntheses",
+        analysis: "analyses",
+        requirement: "requirements",
+        skill: "skills",
+        case: "cases",
+      };
+      const folderMap = { ...builtInFolderMap, ...config.customTypes };
+      const type = params.type as string;
+      const slug = slugify(params.title);
+```
+
+Remove the old `const type = params.type as "entity" | "concept" | ...` cast — it's now just `string` since custom types are valid.
+
+- [ ] **Step 2: Update the type parameter description**
+
+Replace the hardcoded type description (line ~520):
+
+```ts
+      type: Type.String({
+        description:
+          "Page type: entity | concept | synthesis | analysis | requirement | skill | case (built-in) or any user-defined type from llm-wiki.customTypes config",
+      }),
+```
+
+- [ ] **Step 3: Verify buildPageBody fallback**
+
+`buildPageBody` already has a generic fallback at the end (after the `requirement` block) that returns a basic template for any unrecognized type. No code change needed — custom types hit this existing fallback. Just verify it exists by reading the function end.
+
+- [ ] **Step 4: Run existing tests to verify no regression**
+
+Run: `pnpm test`
+Expected: All 699+ tests pass
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add extensions/llm-wiki/lib/tools.ts
+git commit -m "feat(wiki_ensure_page): merge customTypes from config into folder map"
+```
+
+---
+
+## Task 3: Tests for custom types
+
+**Files:**
+- Create: `test/custom-types.test.ts`
+
+- [ ] **Step 1: Write the test file**
+
+```ts
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { registerWikiEnsurePage } from "../extensions/llm-wiki/lib/tools.js";
+import { ensureVaultStructure, getVaultPaths } from "../extensions/llm-wiki/lib/utils.js";
+
+interface Tool {
+  execute: (
+    id: string,
+    params: Record<string, unknown>,
+    s: undefined,
+    u: undefined,
+    ctx: unknown,
+  ) => Promise<{
+    isError?: boolean;
+    content: Array<{ text: string }>;
+    details: Record<string, unknown>;
+  }>;
+}
+
+function capture(fn: (pi: ExtensionAPI) => void): Tool {
+  let tool: Tool | undefined;
+  const pi = {
+    registerTool: (def: unknown) => {
+      tool = def as Tool;
+    },
+  } as unknown as ExtensionAPI;
+  fn(pi);
+  if (!tool) throw new Error("tool not registered");
+  return tool;
+}
+
+let wikiDir: string;
+
+beforeEach(() => {
+  wikiDir = join(
+    import.meta.dirname,
+    "..",
+    "tmp",
+    `ct-${Date.now()}-${Math.random().toString(36).slice(2)}`,
+  );
+  const llm = join(wikiDir, ".llm-wiki");
+  for (const d of ["wiki/entities", "wiki/concepts", "wiki/sources", "meta", "outputs"]) {
+    mkdirSync(join(llm, d), { recursive: true });
+  }
+  writeFileSync(join(llm, "config.json"), JSON.stringify({ topic: "Test", mode: "personal" }));
+  ensureVaultStructure(getVaultPaths(wikiDir));
+  writeFileSync(
+    join(llm, "meta", "registry.json"),
+    JSON.stringify({ version: "1.0", last_updated: "", pages: {} }),
+  );
+});
+
+afterEach(() => {
+  try {
+    rmSync(wikiDir, { recursive: true, force: true });
+  } catch {}
+});
+
+function writeSettings(customTypes: Record<string, string>): void {
+  const cfg = join(wikiDir, ".pi");
+  mkdirSync(cfg, { recursive: true });
+  writeFileSync(
+    join(cfg, "settings.json"),
+    JSON.stringify({ "llm-wiki": { customTypes } }),
+  );
+}
+
+describe("wiki_ensure_page custom types", () => {
+  it("creates a custom type in the correct folder", async () => {
+    writeSettings({ decision: "decisions" });
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "decision", title: "Auth Architecture" },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    const expected = join(getVaultPaths(wikiDir).wiki, "decisions", "auth-architecture.md");
+    expect(existsSync(expected)).toBe(true);
+    expect(res.details.created).toBe(true);
+  });
+
+  it("built-in types still work alongside custom types", async () => {
+    writeSettings({ decision: "decisions" });
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "entity", title: "Test Entity" },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    const expected = join(getVaultPaths(wikiDir).wiki, "entities", "test-entity.md");
+    expect(existsSync(expected)).toBe(true);
+  });
+
+  it("custom type uses generic template when no content provided", async () => {
+    writeSettings({ metric: "metrics" });
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "metric", title: "API Latency" },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    const file = join(getVaultPaths(wikiDir).wiki, "metrics", "api-latency.md");
+    const body = readFileSync(file, "utf-8");
+    expect(body).toContain("# API Latency");
+    expect(body).toContain("## Links");
+  });
+
+  it("custom type with explicit content writes that content", async () => {
+    writeSettings({ decision: "decisions" });
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "decision", title: "Use Postgres", content: "# Use Postgres\n\nWe chose Postgres." },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    const file = join(getVaultPaths(wikiDir).wiki, "decisions", "use-postgres.md");
+    const body = readFileSync(file, "utf-8");
+    expect(body).toContain("We chose Postgres.");
+  });
+
+  it("undefined customTypes (no config) falls back to built-ins only", async () => {
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "concept", title: "Test Concept" },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    const expected = join(getVaultPaths(wikiDir).wiki, "concepts", "test-concept.md");
+    expect(existsSync(expected)).toBe(true);
+  });
+
+  it("unrecognized type without config falls back to concepts folder", async () => {
+    const tool = capture((pi) => registerWikiEnsurePage(pi));
+    const res = await tool.execute(
+      "t",
+      { type: "nonexistent", title: "Fallback Page" },
+      undefined,
+      undefined,
+      { cwd: wikiDir, hasUI: false },
+    );
+    expect(res.isError).toBeFalsy();
+    // folderMap[type] || "concepts" fallback
+    const expected = join(getVaultPaths(wikiDir).wiki, "concepts", "fallback-page.md");
+    expect(existsSync(expected)).toBe(true);
+  });
+});
+```
+
+- [ ] **Step 2: Run the new tests**
+
+Run: `pnpm vitest run test/custom-types.test.ts`
+Expected: All 6 tests pass
+
+- [ ] **Step 3: Run full test suite**
+
+Run: `pnpm test`
+Expected: All tests pass (no regression)
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add test/custom-types.test.ts
+git commit -m "test: custom wiki page types via config"
+```
+
+---
+
+## Task 4: Final verification
+
+- [ ] **Step 1: Typecheck**
+
+Run: `pnpm typecheck`
+Expected: No errors
+
+- [ ] **Step 2: Lint**
+
+Run: `pnpm lint`
+Expected: No errors
+
+- [ ] **Step 3: Full test suite one more time**
+
+Run: `pnpm test`
+Expected: All tests green
+
+- [ ] **Step 4: If all green, push branch and open PR**
+
+```bash
+git push -u origin feat/custom-wiki-page-types
+gh pr create --title "feat: support custom wiki page types via config (closes #169)" --body "..."
+```

package/docs/superpowers/plans/2026-08-29-keyless-provider-ingest-fix.md

@@ -0,0 +1,256 @@
+# Keyless Provider Ingest Fix — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Fix `wiki_ingest` background synthesis silently no-oping for keyless local providers (llama.cpp, self-hosted models).
+
+**Architecture:** One-line logic fix in `Runtime.resolveModel()` + one new test. The `!auth.apiKey` check in the auth gate wrongly rejects providers where `getApiKeyAndHeaders` returns `{ ok: true }` without an `apiKey` field — the correct response for a server that needs no key.
+
+**Tech Stack:** TypeScript, Vitest
+
+**Roadmap:** None
+
+**Phase:** Single-plan implementation
+
+**Issue:** [#174](https://github.com/zosmaai/pi-llm-wiki/issues/174)
+
+---
+
+## Context
+
+`Runtime.resolveModel()` at `extensions/llm-wiki/lib/runtime.ts:135-140` gates background work on API key presence:
+
+```ts
+const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
+if (!auth.ok || !auth.apiKey) {    // ← bug: || !auth.apiKey
+  const provider = (model as { provider?: string }).provider ?? "unknown";
+  return { ok: false, reason: `no API key for provider "${provider}"` };
+}
+return { ok: true, model, apiKey: auth.apiKey, headers: auth.headers };
+```
+
+For keyless providers, the host's `getApiKeyAndHeaders` returns `{ ok: true, headers: {...} }` — `ok: true` because auth is satisfied, no `apiKey` because the server doesn't need one. But `!auth.apiKey` is `true` (undefined is falsy), so `resolveModel` returns `ok: false`.
+
+Downstream in `tools.ts:408`:
+```ts
+const resolved = await runtime.resolveModel(ctx, override);
+if (resolved.ok) {
+  // launch background synthesis ... ← NEVER REACHED for keyless providers
+}
+// falls through to synchronous "📥 N source(s) ready" — no synthesis, no ingest event
+```
+
+The synchronous fallback looks like success to the agent and user, so the bug is invisible.
+
+**The fix:** Remove `|| !auth.apiKey`. The `!auth.ok` clause alone is sufficient — when a provider *requires* auth and has no key, the host returns `{ ok: false }`. When a provider needs no key, `{ ok: true }` without `apiKey` is correct.
+
+---
+
+## Files
+
+| Action | File | Purpose |
+|--------|------|---------|
+| Modify | `extensions/llm-wiki/lib/runtime.ts:135-141` | Fix the auth gate |
+| Modify | `test/runtime.test.ts` — `makeRegistry` + new test | Add keyless provider test |
+
+---
+
+### Task 1: Fix the auth gate in resolveModel
+
+**Files:**
+- Modify: `extensions/llm-wiki/lib/runtime.ts:135-140`
+
+- [ ] **Step 1: Read the current code to confirm the line numbers**
+
+```bash
+awk 'NR>=133 && NR<=143' extensions/llm-wiki/lib/runtime.ts
+```
+
+Expected: lines 135-136 show `if (!auth.ok || !auth.apiKey)` and line 141 shows `apiKey: auth.apiKey`.
+
+- [ ] **Step 2: Apply the fix**
+
+Edit `extensions/llm-wiki/lib/runtime.ts` — replace lines 135-140:
+
+Old:
+```ts
+    const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
+    if (!auth.ok || !auth.apiKey) {
+      const provider = (model as { provider?: string }).provider ?? "unknown";
+      return { ok: false, reason: `no API key for provider "${provider}"` };
+    }
+    return { ok: true, model, apiKey: auth.apiKey, headers: auth.headers };
+```
+
+New:
+```ts
+    const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
+    if (!auth.ok) {
+      const provider = (model as { provider?: string }).provider ?? "unknown";
+      return { ok: false, reason: `no API key for provider "${provider}"` };
+    }
+    return { ok: true, model, apiKey: auth.apiKey ?? "", headers: auth.headers };
+```
+
+Two changes:
+1. `|| !auth.apiKey` removed from the condition (line 136)
+2. `auth.apiKey` → `auth.apiKey ?? ""` on the return (line 140) — defaults to empty string for keyless providers
+
+- [ ] **Step 3: Run typecheck to verify no type errors**
+
+```bash
+pnpm typecheck
+```
+
+Expected: clean exit (no errors). The `apiKey` field is typed `string` in `ResolveResult`; `?? ""` satisfies it.
+
+---
+
+### Task 2: Update tests — existing + new keyless provider case
+
+**Files:**
+- Modify: `test/runtime.test.ts` — `makeRegistry` helper + new test
+
+- [ ] **Step 1: Read the existing test to confirm it still applies**
+
+```bash
+awk 'NR>=176 && NR<=181' test/runtime.test.ts
+```
+
+Expected: the test uses `authOk: false` — this tests "provider requires auth but has no key" → should still be rejected. This test is correct and stays unchanged.
+
+- [ ] **Step 2: Fix `makeRegistry` to support explicit `apiKey: undefined`**
+
+The current `makeRegistry` defaults `apiKey` to `"key-123"` when `authOk: true`, so passing `apiKey: undefined` doesn't produce the keyless scenario. Fix the helper to use `"apiKey" in opts` to distinguish "not passed" from "explicitly undefined":
+
+Old:
+```ts
+function makeRegistry(opts: {
+  found?: unknown;
+  authOk?: boolean;
+  apiKey?: string;
+}) {
+  return {
+    find: (_p: string, _i: string) => opts.found,
+    getApiKeyAndHeaders: async (_m: unknown) => ({
+      ok: opts.authOk ?? true,
+      apiKey: opts.apiKey ?? (opts.authOk === false ? undefined : "key-123"),
+      headers: { "x-test": "1" },
+    }),
+  };
+}
+```
+
+New:
+```ts
+function makeRegistry(opts: {
+  found?: unknown;
+  authOk?: boolean;
+  apiKey?: string | undefined;
+  _apiKeySet?: boolean; // true when caller explicitly passed apiKey
+}) {
+  const apiKeyExplicit = opts._apiKeySet ?? false;
+  return {
+    find: (_p: string, _i: string) => opts.found,
+    getApiKeyAndHeaders: async (_m: unknown) => ({
+      ok: opts.authOk ?? true,
+      // When apiKey was explicitly passed (even as undefined), use it as-is.
+      // When omitted, default: "key-123" for auth-ok, undefined for auth-fail.
+      apiKey: apiKeyExplicit ? opts.apiKey : (opts.authOk === false ? undefined : "key-123"),
+      headers: { "x-test": "1" },
+    }),
+  };
+}
+```
+
+And update all existing call sites to pass `_apiKeySet: false` (or omit it — the default handles existing tests). Only the new keyless test passes `_apiKeySet: true`.
+
+- [ ] **Step 3: Add the new keyless provider test**
+
+After the existing "returns ok:false when the provider has no API key" test (line 181), add:
+
+```ts
+  it("accepts a keyless provider (authOk, no apiKey)", async () => {
+    const rt = new Runtime();
+    const reg = makeRegistry({ found: undefined, authOk: true, apiKey: undefined, _apiKeySet: true });
+    const res = await rt.resolveModel({ model: SESSION_MODEL, modelRegistry: reg, hasUI: false });
+    expect(res.ok).toBe(true);
+    if (res.ok) {
+      expect(res.model).toBe(SESSION_MODEL);
+      expect(res.apiKey).toBe("");
+      expect(res.headers).toEqual({ "x-test": "1" });
+    }
+  });
+```
+
+This tests the exact scenario from #174: `getApiKeyAndHeaders` returns `{ ok: true, apiKey: undefined }` (keyless provider). The fix should make this pass.
+
+- [ ] **Step 3: Run the test suite**
+
+```bash
+pnpm test
+```
+
+Expected: all tests pass, including the new keyless provider test. The existing "no API key" test (authOk: false) still passes — it tests a different case.
+
+- [ ] **Step 4: Run the full gate (typecheck + lint + test)**
+
+```bash
+pnpm typecheck && pnpm lint && pnpm test
+```
+
+Expected: all green.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add extensions/llm-wiki/lib/runtime.ts test/runtime.test.ts
+git commit -m "fix: accept keyless providers in resolveModel (closes #174)
+
+remove the !auth.apiKey guard that wrongly rejected providers where
+getApiKeyAndHeaders returns ok:true without an apiKey field (the correct
+response for keyless local servers like llama.cpp). apiKey defaults to
+\"\" for keyless providers, which local servers ignore.
+
+adds a test for the keyless provider case (authOk, no apiKey)."
+```
+
+---
+
+### Task 3: Verify end-to-end
+
+- [ ] **Step 1: Run the full test suite one final time**
+
+```bash
+pnpm test
+```
+
+Expected: 725+ tests pass (725 from last session + 1 new = 726).
+
+- [ ] **Step 2: Check git diff to confirm minimal change**
+
+```bash
+git diff HEAD~1 --stat
+```
+
+Expected: 2 files changed, ~3 lines removed, ~10 lines added (test).
+
+- [ ] **Step 3: Verify no other callers of resolveModel are affected**
+
+```bash
+grep -rn "resolveModel" extensions/ --include="*.ts"
+```
+
+Expected: only `runtime.ts` (definition) and `tools.ts` (caller). Both are safe — `tools.ts` already handles `ok: true` with empty `apiKey` via `runSubAgent`.
+
+---
+
+## What was skipped
+
+- **Docs update:** No user-facing docs change needed — this is a bug fix, not a feature. The `docs/configuration.md` already documents `taskModel` configuration. The fix makes the documented behavior actually work for keyless providers.
+- **Changelog entry:** Will be captured in the PR description and release notes when merged.
+
+## When to add
+
+- **Robustness:** If a provider returns `{ ok: true, apiKey: "" }` explicitly (not just undefined), the `?? ""` still handles it correctly.
+- **End-to-end test with real llama.cpp:** Not feasible in CI (needs a running local server), but the unit test covers the exact code path. Manual verification: point pi at a keyless local provider, run `wiki_capture_source` + `wiki_ingest`, confirm background synthesis launches.