@moikapy/lich 0.3.0

package/docs/architecture/tools.md

@@ -0,0 +1,180 @@
+# Tools
+
+Tools are how the model acts on the world. The contract is deliberately tiny:
+a name, a description, a JSON Schema for arguments, and an async function that
+**never throws**. Everything else - confinement, timeouts, clamping, error
+shaping - is layered on top by the guard helpers and the executor.
+
+## The tool interface contract
+
+```ts
+export interface ToolResult {
+  ok: boolean;
+  output: string;
+  error?: string;
+}
+
+export interface ToolContext {
+  work_dir: string;
+  env: Record<string, string>;
+  signal?: AbortSignal;
+}
+
+export interface Tool {
+  name: string;
+  description: string;
+  parameters: JsonSchemaObject;
+  execute(args: Record<string, unknown>, context: ToolContext): Promise<ToolResult>;
+}
+```
+
+(src/tools/types.ts)
+
+**Never-throw convention.** `execute` resolves with a `ToolResult`; it reports
+failure through `ok: false` plus an `error` string. Throwing is tolerated -
+`capture_errors` wraps every builtin body and the executor catches the rest -
+but the convention is to return, not throw, so callers get a well-shaped
+result either way.
+
+**`ToolContext`** gives each execution a working directory (`work_dir`, the
+confinement root), a process environment map (the agent injects
+`LICH_TERMINAL_TIMEOUT_MS`), and an abort `signal` that fires on caller abort
+**or** the executor's own 30 s deadline.
+
+**Parameter schemas.** `parameters` is a `JsonSchemaObject`
+(`src/util/json_schema.ts`) passed through verbatim into provider requests.
+The wire-format keys (`properties`, `required`, `additionalProperties`) are a
+**deliberate exemption** from the repo's snake_case convention, noted in the
+source: these objects are serialized into LLM requests, so they must keep the
+JSON Schema spelling.
+
+## Guardrails deep dive
+
+[`src/tools/guard.ts`](../../src/tools/guard.ts) holds the safety helpers
+every builtin composes.
+
+### `resolve_safe_path` - confinement math
+
+```ts
+export function resolve_safe_path(base_dir: string, target: string): string {
+  const base = path.resolve(base_dir);
+  const resolved = path.resolve(base, target);
+  const relative = path.relative(base, resolved);
+  if (relative.startsWith("..") === true || path.isAbsolute(relative) === true) {
+    throw new Error(`path_escape: ${target} escapes ${base_dir}`);
+  }
+  return resolved;
+}
+```
+
+(src/tools/guard.ts)
+
+Both paths are resolved first (so symlinks are *not* the defense here - this
+is lexical confinement), then `path.relative` computes the containment
+relationship: escaping the base always produces a relative path starting with
+`..`, and the absolute check catches edge cases such as different Windows
+drive letters. Absolute targets are accepted as long as they land inside the
+base. All filesystem builtins (`read_file`, `write_file`, `edit_file`,
+`list_dir`, `grep_files`, `disk_usage`) confine through this function; the
+error prefix `path_escape:` is the model-visible signal.
+
+### `with_timeout` - deadline race
+
+`with_timeout(promise_factory, timeout_ms, label)` builds its own
+`AbortController`, races the factory's promise against a deadline promise, and
+aborts the controller when the deadline fires. Key details:
+
+- The factory receives the controller's signal, so long-running work (child
+  processes, reads) can react to the deadline.
+- The `setTimeout` handle is always cleared in a `finally`, so no timer leaks
+  even when the work wins the race.
+- Losing the race rejects with `ToolTimeoutError` (message:
+  `timeout: <label> exceeded <n>ms`).
+- The raced work promise gets a `.catch(() => undefined)` so a late failure
+  does not surface as an unhandled rejection.
+
+### Clamping and coercion
+
+- `clamp_output(text, max_chars = 20000)` truncates via `truncate_text`,
+  appending `[... truncated, N chars omitted ...]` - bounds on what a tool can
+  pour into the context.
+- Argument coercion helpers make malformed LLM args non-fatal:
+  `require_string_arg` (throws `missing_arg: <key>`), `optional_string_arg`,
+  `optional_number_arg`, `optional_boolean_arg` - each falls back on
+  absent/empty/wrong-typed values.
+- `is_enoent` detects Node fs ENOENT without subclass checks, and
+  `error_result` / `capture_errors` convert any thrown value into
+  `{ ok: false, output: "", error: message }`.
+
+## Executor semantics
+
+[`ToolExecutor`](../../src/tools/executor.ts) never throws. `execute(name,
+args, context?)`:
+
+1. **Unknown tool** -> error result, not an exception:
+   `{ ok: false, output: "", error: "unknown_tool: <name>" }`. The model can
+   read the message and self-correct.
+2. **Default context** when none is passed: `work_dir` from the executor
+   defaults (falling back to `process.cwd()`) plus the configured `env`.
+3. **Cancelled fast path**: if the context signal is already aborted, return
+   `{ ok: false, output: "", error: "cancelled" }` without running the tool.
+4. **Signal merge + 30 s timeout**: a fresh `AbortController` is aborted by
+   the external signal (an `abort` listener), by the `with_timeout` deadline
+   (`DEFAULT_TOOL_TIMEOUT_MS = 30000`), and the merged signal is what the tool
+   receives. The external listener is removed in a `finally`.
+5. **Output clamping**: successful results pass through `clamp_result`
+   (`clamp_output`, 20 000 chars).
+6. **Failure shaping**: any throw (including `ToolTimeoutError`) lands in
+   `failure_result`, which returns `error: "cancelled"` if the abort fired,
+   else `error_result(err)`.
+
+`ToolExecutor.format_result(result)` renders a result for a tool-role message:
+**raw output on success, JSON on error**
+(`{"ok":...,"output":...,"error":...}`). The loop mirrors this exact convention
+in `format_tool_result_content` when building `ToolMessage`s (src/agent/loop.ts)
+- the two stay in sync by convention, which is why the TUI can parse either
+form back with `parse_tool_message_content` (src/tui/state.ts).
+
+## Builtin catalog
+
+Twelve tools, registered by `register_builtin_tools`
+([`src/tools/builtin/index.ts`](../../src/tools/builtin/index.ts)):
+
+| Tool | Key args | Implementation insight |
+| --- | --- | --- |
+| `read_file` | `path`, `offset?`, `limit?` | 1-based line slicing, 256 KB clamp; ENOENT becomes `not_found:` error result, not a throw. |
+| `write_file` | `path`, `content` | `mkdir` on the parent first, so new directories come for free; reports char count. |
+| `edit_file` | `path`, `old_string`, `new_string`, `replace_all?` | Fails `old_string_not_found` / `old_string_not_unique (N)` unless `replace_all` - an exact-match protocol that forces the model to anchor edits. |
+| `list_dir` | `path?`, `depth?` (1-4) | Iterative worklist (no recursion), dirs-first sorting, skips `node_modules`/`.git`/`dist`/`.lich`/`.cursor`, caps at 500 entries, file sizes via `stat`. |
+| `terminal` | `command`, `timeout_ms?` | Spawns `bash -lc`, streams and caps stdout+stderr at 50 K chars, SIGKILLs on deadline, appends `[exit N]`; `ok` requires exit code 0 and no cancellation. |
+| `grep_files` | `pattern`, `path?`, `glob?`, `max_results?` | Explicit stack walk (no recursion), binary sniff (NUL byte in first 1000 bytes), 1 MB file cap, `*.ext` suffix-glob matcher, overcollect-by-one to report suppressed counts. |
+| `fetch_url` | `url`, `max_chars?`, `timeout_ms?` | GET only; rejects non-http(s) protocols; refuses images/octet-stream; tags HTML bodies with `[html content]`; status/type header line first. |
+| `web_search` | `query`, `max_results?` | Scrapes DuckDuckGo's HTML endpoint (no API key); unwraps `uddg=` redirect links; decodes the handful of entities DDG emits. |
+| `http_request` | `url`, `method?`, `headers?`, `body?`, ... | Method allowlist (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS); stringified caller headers; reports `content-length`, `ratelimit-remaining`, `retry-after`. |
+| `process_list` | `filter?`, `max_results?` | Reads `/proc` synchronously: numeric dirs are pids, `cmdline` is NUL-separated; missing entries (process died mid-scan) read as empty. |
+| `disk_usage` | `path?`, `max_entries?` | One `du -sb` subprocess per depth-1 entry with a 10 s timeout; sorted desc with a `TOTAL` row; `du` missing yields `du_unavailable`. |
+| `env_get` | `keys?`, `prefix?`, `reveal?` | Values are hidden unless `reveal`; names matching `/(secret\|token\|password\|key\|credential\|auth)/i` are **always** masked as `<redacted: N chars>`. |
+
+The three HTTP tools (`fetch_url`, `web_search`, `http_request`) share
+helpers from `fetch_url.ts`: `valid_http_url` (URL parse + protocol
+allowlist), `compose_abort_signal` (per-call `AbortSignal.timeout` merged
+with the executor's cancellation via `AbortSignal.any`), and `clamp_int_arg`
+(floored, bounded to `[1, max]`).
+
+## Registry
+
+[`ToolRegistry`](../../src/tools/registry.ts) is a name-keyed `Map`:
+
+- **Duplicate rejection**: `register` throws
+  `duplicate_tool: <name>` if the name exists - conflicting builtins fail
+  loudly at startup instead of silently shadowing.
+- **`definitions()`** maps each registered tool onto the provider-facing
+  `ToolDefinition` shape (`name`, `description`, `parameters`).
+- **`register_toolset`** registers a named group (`Toolset`) at once; the
+  builtins ship as `builtin_toolset`.
+- **Enabling a subset** is done by rebuilding a fresh registry: `Agent`'s
+  `filter_registry` (src/agent/agent.ts) iterates `base.list()` and registers
+  only allowed names onto a new `ToolRegistry` when `tools_enabled` is a list
+  (`"all"` returns the base registry untouched).
+
+For building your own tool, see [extending](./extending.md#add-a-builtin-tool).

package/docs/design/council/architecture-review.md

@@ -0,0 +1,47 @@
+# Council Review — Architecture & Testability
+
+Reviewer: Council Member 3 (architecture fit, correctness, testability)
+Plan under review: `docs/design/self-improvement-loop.md` v1
+
+## Verdict
+
+APPROVE-WITH-CHANGES. The loop's shape fits the plugin/hook seam well, but the plan hand-waves four load-bearing contracts: the executor's hard 30s tool timeout (which makes the 10-minute `run_tests` impossible as specced), per-run gatekeeper state (`HookContext` cannot carry it, and loop-driven hooks see `process.cwd()`, not `work_dir`), the consolidation `ChatFn` plumbing (plugins never receive config today), and M4's same-run self-registration claim (impossible under construction-time registration plus the module cache).
+
+## Blocking findings
+
+1. **Executor hard-caps every tool call at 30s — the 10-min `run_tests` cap is unreachable — HIGH.** `ToolExecutor.run_tool` wraps every tool in `with_timeout(..., DEFAULT_TOOL_TIMEOUT_MS, ...)` and `DEFAULT_TOOL_TIMEOUT_MS = 30000` (`src/tools/executor.ts:53-70`, `src/tools/guard.ts:8-9`). Vitest on this NFS mount takes 170s+; `run_tests` would die at 30s with `timeout: tool:run_tests exceeded 30000ms`. (Side effect: the `terminal` tool's own 300s max, `src/tools/builtin/terminal.ts:14-16`, is already dead beyond 30s under the executor — a latent bug worth a separate fix.) Fix: add an optional `timeout_ms` to the `Tool` interface, or an `ExecutorDefaults` timeout fed from a new config key, and use it in `run_tool`.
+2. **Gatekeeper state has nowhere to live — HIGH.** `HookContext` is `{ work_dir: string }` only (`src/plugins/types.ts:8-10`). Worse: the loop calls `deps.tools.execute(call.name, call.args)` with no context (`src/agent/loop.ts:87-101`), so `HookedToolRunner.execute` builds hook ctx as `{ work_dir: process.cwd() }` (`src/plugins/hooks.ts:89-97`) — during real runs, before/after hooks see `process.cwd()`, NOT `config.work_dir` (only `on_run_start`/`on_run_end` get `config.work_dir`, `src/agent/agent.ts:166-180`). The plan's "tracked via `on_run_start` reset state" assumes a per-run state channel that does not exist, and work_dir-keyed module state would mis-key whenever cwd ≠ work_dir, plus breaks under concurrent runs in one process. Fix: extend `HookContext` with run identity / a mutable per-run bag (e.g. `run_id`, `state`), built in `Agent.run` and threaded through `hooks.ts` — a type change to `plugins/types.ts` + `hooks.ts` + `agent.ts` the plan must name explicitly.
+3. **`on_run_end` is not reliable on the paths the reflector cares about — HIGH.** `Agent.run` fires `call_plugin_run_end` only `if (outcome !== undefined)` inside `finally` (`src/agent/agent.ts:133-149`). The loop returns normally with `stopped_reason` ∈ {final, budget, aborted-between-turns} (`src/agent/loop.ts:172-207`), but a throw mid-LLM (provider error, abort during chat) leaves `outcome === undefined` and skips `run_end` entirely. Also, `persist_session` runs AFTER the run_end fan-out (`src/agent/agent.ts:149-157`), so a reflector reading the session JSONL in `on_run_end` sees only the previous run's file. Fix: reflector must count tool calls/failures via its own `after_tool_call` bookkeeping (only a 300-char `result_summary` is available, `src/plugins/hooks.ts:15-18`), and the spec must either move persistence before fan-out or accept lossy recording on crash paths.
+4. **Consolidation `ChatFn` plumbing is unspecified — HIGH.** The agent's chat fn is a private closure over `ProviderRouter` (`src/agent/agent.ts:158-165`); nothing is exposed. Plugins are dynamic-imported modules whose extracted object carries only `name`/`tools`/`hooks` (`src/plugins/loader.ts:48-60`); no config is ever handed to a plugin (`create_agent_with_plugins`, `src/agent/agent.ts:209-215`). A reflector plugin cannot obtain any `ChatFn` today. Fix (pick one and spec it): (a) optional `configure(config: AgentConfig)` on `Plugin`, called at construction, letting the reflector build its own `ProviderRouter` (config is deep-frozen, `src/agent/config.ts:49-59`, which is fine for this); or (b) a public chat-fn accessor on `Agent`. Q4 is unanswerable until this seam exists.
+5. **M4's "registers → uses it same-run" is impossible — HIGH.** The registry is built once in the `Agent` constructor (`src/agent/agent.ts:108-114`) via `register_builtin_tools` (`src/tools/builtin/index.ts:52-57`), and the plugin loader dynamic-imports only at startup (`src/plugins/loader.ts:80-100`). Bun caches imported modules, so the agent editing `src/tools/builtin/index.ts` cannot alter its own running registry, and there is no runtime re-registration path. Fix: redefine M4 honestly — run 1 validates write + `run_tests` green + gated `git_commit`; the e2e then spawns a FRESH process (second `Agent` or CLI subprocess) to assert the new tool registers and executes. Same-run usage would require plugin hot-loading, which is out of v1 scope.
+6. **Concurrent `run_tests` collide on one repo — MEDIUM.** `ToolExecutor` provides no cross-run serialization, and two vitest pools on the same NFS checkout fight over worker pools and temp dirs. Fix: an `O_EXCL` lock file at `<work_dir>/.lich/locks/run_tests.lock` with stale-age eviction, or a process-level promise chain inside the gatekeeper plugin — spec which, and the failure message when the lock is held.
+7. **Skills over two roots vs. single-slot memoized caches — MEDIUM.** `docs_read` memoizes `cached_root`/`cached_files` as module globals and its resolver is first-resolution-wins (`src/tools/builtin/docs_read.ts:18-19, 88-104, 166-174`); `docs_search` keeps its own single-slot `cached_sections` keyed to one root (`src/tools/builtin/docs_search.ts:19-22, 78-86`). Parameterizing "the same engine" over docs+skills thrashes these caches, and — critically — a `skill_save` in the same run would be invisible to `skill_search` until restart because the file list is memoized. Fix: per-root `Map` caches (bounded), a separate skill resolver (same seam shape as `set_docs_root_resolver`), and explicit invalidation after `skill_save`.
+
+## Non-blocking observations
+
+- "Refuses unrelated staged changes" needs no index plumbing: `git diff --cached --name-only` for the guard, then `git commit --only -m <msg> -- <paths>` stages exactly the named paths atomically.
+- Turn budget is a non-issue: M4 needs ~6-8 LLM turns (tool calls batch within a turn); `max_turns=25` suffices (`src/agent/config.ts:29`). Wall clock is the real constraint: 170s+ vitest × retries plus slow NFS `git status` — raise the e2e `testTimeout` to ~20 min.
+- NFS `.git` is already flaky here (`.git/tkfNobv: Function not implemented` observed during review). Build e2e fixture repos on local disk (`/tmp`), not `test/.tmp/` on the NFS mount.
+- `RunEndInfo` carries only `stopped_reason`/`turns_used` (`src/plugins/types.ts:31-34`); the reflector's "≥3 tool calls" heuristic needs hook-side counting or a type extension.
+- `tools_enabled` exclusion already removes `git_commit` from the registry (`src/agent/agent.ts:42-57`), so the gatekeeper's exclusion check is redundant-but-harmless depth.
+- Plan's default `test_command` (`node node_modules/vitest/vitest.mjs run`) conflicts with repo practice (`bun test` / `bun x vitest run`, `docs/architecture/extending.md`); make the default configurable and document the NFS-timeout reality.
+- Vetoed calls surface to the model as `blocked_by_plugin: <reason>` tool errors (`src/plugins/hooks.ts:93-97`) — good feedback loop; spec the exact reason strings.
+- Memory/skills-in-prompt are static per run; edits apply next run — consistent with the "verify before acting" framing, fine.
+- Generalized skill search inherits the iterative-stack walker (no recursion), matching repo convention.
+
+## Answers to the plan's open questions
+
+- Q1: Restrict `run_tests` to `config.work_dir` in v1; an explicit-directory mode is a later opt-in flag, not a default.
+- Q2: Explicit `skill_search` only; description-triggered auto-load is nondeterministic prompt churn and should be a future opt-in.
+- Q3: `max_commits_per_run = 1` is the right default; multi-commit work should mean multiple runs (human checkpoints), keep the value config-tunable.
+- Q4: Dedicated `consolidation_provider` slot defaulting to the main chain — but only meaningful after finding #4's config plumbing is specced.
+- Q5: YAGNI — keep `created` + `source_run` in v1; add `version` only when overwrite conflicts actually occur.
+
+## Required spec additions
+
+1. `HookContext` v2: run identity + per-run state bag (or an explicit module-state contract with reset semantics), uniform work_dir semantics across lifecycle vs. tool hooks, and behavior under concurrent runs.
+2. Executor timeout seam: where `run_tests`' 10-min cap lives (per-tool `timeout_ms` on `Tool` vs. config-driven `ExecutorDefaults`), plus the latent terminal-tool 30s-cap mismatch.
+3. `run_tests` contract: cwd = `config.work_dir`; `test_command` zod key in `src/agent/config.ts`; machine-readable failure extraction (vitest reporter choice); lock/serialization policy; structured `{passed,total,failures[]}` rendering.
+4. `git_commit` primitive spec: guard via `git diff --cached --name-only`, commit via `git commit --only -- <paths>`, e2e env isolation (`GIT_CONFIG_NOSYSTEM=1`, disposable `HOME`/`XDG_CONFIG_HOME` seeded with `user.email`/`user.name`), never-push invariant, NFS-slowness notes.
+5. Consolidation plumbing: chosen config-injection mechanism, provider slot, best-effort failure policy inside `on_run_end` (never throws), and concurrent-append policy for `candidates.md`.
+6. M4 revised flow: two-process verification of registration, mock-provider wiring through the same public seam the demo uses, e2e timeout budget, and fixture-repo git isolation commands.

package/docs/design/council/security-review.md

@@ -0,0 +1,39 @@
+# Council Review — Security & Safety
+
+## Verdict
+
+**REJECT.** The plan's central safety claim — "the veto cannot be bypassed by the agent because hooks wrap the executor" — is factually false against the actual code: hooks wrap `ToolExecutor.execute` (src/plugins/hooks.ts:88-104), but the builtin `terminal` tool runs arbitrary `bash -lc` (src/tools/builtin/terminal.ts:69), so any name-keyed veto (`git_commit`, config writes, plugin edits) is bypassed in one shell call. The loop is salvageable, but the gatekeeper must be redesigned from "in-process veto" to "fail-closed integrity + human gate", and v1 memory injection into the system prompt must be cut.
+
+## Blocking findings
+
+1. **Gatekeeper bypass via `terminal`** — HIGH. `terminal` executes any command with exit-code capture and no command inspection (terminal.ts:69, 104-112). A veto keyed on `git_commit` is defeated by `terminal("git commit -m ...")`, `git add -f .env`, `bun run build` (overwrites `dist/`, which `loader.ts:67-69` dynamic-imports — code replacement without any tool call), or killing/rewriting anything under `.lich/`. Fix: implement `git_commit` via git plumbing, never shell; when `allow_self_commit` runs are active, veto `terminal` commands matching `git (add|commit|push)` and writes to `.lich/` (best-effort — bash obfuscation defeats denylists, so this raises cost, it does not close the hole). Honest note: fully closing it conflicts with keeping `terminal` useful; the real floor is the human reviewing the commit. Tension with the 60-line-helper rule: a shell-command scanner wants to be its own module.
+
+2. **The gate is fail-open and self-deletable** — HIGH. `load_plugins` never throws; broken/missing entries are collected, warned, and skipped (loader.ts:99-101), and `agent.ts:209-215` only logs. The agent can delete its plugin entry from `.lich/config.json` or `rm` the plugin file with `write_file`/`terminal` (both confined only to work_dir — guard.ts:12-20 — and the plugin/config live *inside* work_dir); next startup runs ungated, silently. Fix: hash-pin the gatekeeper entry and config plugin list outside the agent-writable tree (e.g. XDG state dir); on mismatch/absence, refuse to start *or* refuse to expose `git_commit`/`run_tests` at all (hard-fail beats warn). Note tension: hard startup failure annoys normal users who disable plugins on purpose — scope the hard failure to self-improvement tools only.
+
+3. **The "tests ran in same run" state is forgeable and stale (TOCTOU)** — HIGH. The gate tracks "a `run_tests` call succeeded in the same run" (plan §3); an agent can run tests on clean code, edit source, then commit — the seal never re-checks the tree. It also can't see outcomes: `after_tool_call` gets a 300-char clamp of `error ?? output` (hooks.ts:23, 99-101), so structured `{passed, total, failures[]}` truncates and a "pass" prefix is all the hook sees. Fix: the gatekeeper seals `{run_id, HEAD, hash("git status --porcelain"), tests_ok}` at test time into a file outside work_dir, and `before_tool_call` on `git_commit` re-derives the current porcelain hash and vetoes on mismatch. That kills run-across-edit staleness; same-run edits after a green test still require a re-run.
+
+4. **MEMORY.md in the system prompt is persistent prompt injection** — HIGH. First 40 lines of a file the agent's own tools write are injected into every future run's system prompt. Poison planted via a tool result this run → candidates.md → consolidation LLM → MEMORY.md steers all later runs, with no human in that path (plan §5). "The system prompt marks it as notes" is a wish, not a control. Fix for v1: do **not** put memory in the system prompt; expose `memory_read` as a tool and keep writes human-reviewed (or quarantined pending review). Sanitizing prose is not reliable — don't pretend otherwise.
+
+5. **Skills/skill_save are injected-influenced data with no trust boundary** — MED/HIGH. "Skills are DATA, not instructions" is asserted in the plan text, not enforced anywhere in the reviewed code — `docs_read`/`docs_search` return raw file text as tool output (docs_read.ts:195, 231), and skills read the same way. A poisoned skill file is indistinguishable from a doc the model will follow. Docs get human review via commits; agent-written skills don't. Fix: v1 skills are written only through the commit path (human-reviewed), or are read-only to the agent and auto-never-loaded (see Q2).
+
+6. **Reflector write path has no schema or failure contract** — MED. Consolidation is a plain ChatFn call; `ProviderError` kinds (rate_limit/network/auth/overflow/bad_request, providers/types.ts:9-15) are unspecified here, and the compressor precedent is warn-and-keep-old (loop doc "best-effort fallback"). But this LLM's output is *written to disk* as memory/skill — a truncated or injected reply becomes durable state. Fix: dedicated provider slot (Q4: yes), no tools attached to the call, strict output schema (≤200 chars/line, no URLs), failures leave candidates.md untouched.
+
+7. **Plugin tools bypass `tools_enabled`** — MED. `filter_registry` applies the allowlist to builtins only; `register_plugin_tools` registers plugin tools afterwards (agent.ts:47-56, 72-83). A gatekeeper shipped as a plugin cannot be disabled by the user via config, and once finding #2's write access exists, a swapped plugin gets the full registry. Fix: apply the allowlist to plugin tools too, or ship the gatekeeper in core.
+
+8. **Concurrent runs corrupt the memory store** — MED. `memory_write` appends while consolidation rewrites MEMORY.md read-modify-write; two runs → lost lines, torn files, duplicate skills (NFS makes this worse — the gvfs mount already saw a suite-file cleanup flake). Fix: per-run candidate files `candidates/<run_id>.md` (append-only, no cross-writer contention); consolidation is the single writer, guarded by an exclusive lockfile.
+
+## Non-blocking observations
+
+- `git_commit` staging: partial staging (agent stages file A, misses helper B) passes a seal keyed on "tests green earlier"; require the tool to stage exactly the named set and re-run tests post-stage, or commit on a dedicated `agent/*` branch only (never current/main) — also answers wrong-branch and non-lich-repo (`work_dir` is config-driven; require the commit target repo to be the pinned lich repo).
+- `git add -f .env` style secret leaks remain possible via terminal even with the seal; mitigations are best-effort vetoes (finding 1), so pre-commit `git diff --cached` review in the veto reason is worthwhile.
+- `spawn("bash", ["-lc", ...])` inherits full `process.env` (terminal.ts:69) and `env_get` exists; combined with `fetch_url` this is a pre-existing exfiltration path the loop makes more valuable to exploit.
+- Session JSONL under `.lich/sessions` is agent-writable; transcripts are forgeable "history" — keep out of any trust decision.
+- `max_commits_per_run=1` (Q3) is fine as a blast-radius cap but is a speed bump, not a gate; per-run reset state is only as trustworthy as finding 3's seal.
+
+## Answers to the plan's open questions
+
+- **Q1**: Restrict `run_tests` to the pinned lich work_dir (git-repo check); generalizing it turns a safety signal into an arbitrary-code launcher in any directory.
+- **Q2**: Only explicit `skill_search`/`skill_read`; auto-load would make file descriptions (writable by the agent) injected prompt content — persistent injection by another name.
+- **Q3**: 1 commit/run is reasonable as a cap, meaningless as safety; the seal (finding 3) is the actual control.
+- **Q4**: Dedicated cheaper provider slot, tools-free, schema-parsed — also isolates cost and prompt-injection surface from the main chain.
+- **Q5**: YAGNI for v1 — skills written only via human-reviewed commits make versioning redundant; revisit only if agent-written skills ever ship.

package/docs/design/council/simplicity-review.md

@@ -0,0 +1,45 @@
+# Council Review — Simplicity & Value
+
+Reviewer: Council Member 2 (Simplicity / YAGNI / Value) — review of `docs/design/self-improvement-loop.md` v1.
+
+## Verdict
+
+APPROVE-WITH-CHANGES — The valuable core (implement → `run_tests` → gated `git_commit`) is small and rides on machinery that already exists. But the plan ships a parallel skills subsystem (three rebadged docs tools), an unrequested LLM consolidation pipeline, a redundant `repo_status`, and four milestones for a one-maintainer feature — cut about two-thirds of it.
+
+## Blocking findings
+
+1. **Skills are docs rebadged (M2) — high.** `skill_search`/`skill_read` are `docs_search`/`docs_read` (`src/tools/builtin/docs_search.ts`, `docs_read.ts`) pointed at a second directory, and `skill_save` adds hand-rolled YAML frontmatter validation for metadata (`version`, `source_run`) nothing consumes in v1. The docs resolver is already a swappable seam (`set_docs_root_resolver` in `docs_read.ts`, built for tests); teaching it a second root like `.lich/skills` is ~10 lines. Simpler: **skills are `.md` files under `.lich/skills/`**, searchable/read by the existing tools, written by the agent with `write_file` (which already mkdirs parents — `src/tools/builtin/write_file.ts`). Zero new tools; a `# title` + date line is data enough, no frontmatter validator to maintain.
+
+2. **LLM consolidation pass is auto-ML nobody asked for — high.** The reflector's every-N-runs ChatFn curation (plan §5) buys a second LLM call path, an N-run counter with unspecified persistence (in-memory state silently resets on restart), an unbounded `candidates.md` lifecycle, and a plugin that silently writes skill files — contradicting the plan's own "propose, don't ship". The simplest loop-closer: the agent appends one dated lesson line to `.lich/memory/MEMORY.md` itself as the last step of a self-improvement run (200-line cap, oldest dropped — a tiny append helper, no tool needed). Curation = the human deletes stale lines, or an explicit user-invoked "tidy memory" prompt later. No background pass, no counter, no Q4.
+
+3. **`repo_status` is `terminal git status` in a costume — medium.** `terminal` is already core (`src/tools/builtin/index.ts`). A dedicated tool is schema + tests + docs for zero capability delta, and nothing gates on it (unlike `run_tests`). Cut it; document a one-recipe git workflow in the guide.
+
+4. **`git_commit` earns its place — say why, then keep it — medium.** Vetoing free-form `terminal git commit/push` strings is brittle pattern-matching the agent can dodge (quoting, alternate paths); a named tool is the one stable choke point a `before_tool_call` veto can enforce (`src/plugins/hooks.ts`). So the gatekeeper is fine as specified — it is a plain consumer of the documented hook semantics (`docs/user-guide/plugins.md`) and serves the safety mandate. This is the single defensible new tool; everything else is optional.
+
+5. **Four milestones for a solo maintainer — medium.** Only M4 proves the claim "Lich codes Lich"; M1–M3 are merely its parts. Three intermediate releases is process overhead on a repo at `0.3.0` with one `test` script (`package.json`). Ship one release whose acceptance test *is* the M4 e2e.
+
+## Non-blocking observations
+
+- Token economics: injecting MEMORY.md into the system prompt every run (plan §4) taxes every trivial run to save the agent one `docs_search`. Keep memory a plain searchable file, loaded on demand — zero plumbing.
+- `run_tests` keeps structured `{passed, total, failures[]}`: it is the machine-checkable gate signal; parsing terminal transcripts is not. (Adversarial tension: green-tests-only gating can push the agent to delete failing tests — the human commit review is the real control; say so in docs.)
+- Conventions: only the consolidation prompt-parsing + plugin-writes-skill-files design strains the 60-line hook rule; everything in the minimal version fits. Existing code is compliant (docs walker is iterative, `docs_read.ts`).
+- If skills join the docs tools, the single-slot section cache in `docs_search.ts` (keyed on one root) must key per-root — a few lines, note it in the resolver change.
+- Hidden-cost inventory otherwise checks out: no new dependencies, no recursion, `test_command` default matches the `package.json` test script.
+
+## Answers to the plan's open questions
+
+- **Q1:** Restrict `run_tests` to `work_dir` in v1; cross-repo runs are `terminal` territory.
+- **Q2:** Explicit search only. Description-match auto-loading is speculative ranking machinery.
+- **Q3:** 1 is right for "propose, don't ship" — and don't make it configurable; hardcode until proven wrong.
+- **Q4:** Moot — cut consolidation. If it ever returns, main provider chain; a second config slot is YAGNI.
+- **Q5:** YAGNI. A `created` date line suffices; drop `version` and `source_run`.
+
+## Proposed minimal version
+
+Smallest subset that still achieves "Lich codes Lich":
+
+- **Tools (2):** `run_tests` (structured pass/fail + failing names, timeout, config `test_command`), `git_commit` (stages named paths, never pushes).
+- **Plugins (1):** gatekeeper — veto `git_commit` without an in-run green `run_tests`, max 1 commit/run, `allow_self_commit: false` default. No reflector plugin.
+- **Skills (0 tools):** `.lich/skills/*.md` written with `write_file`, found via existing docs search/read after the ~10-line second-root resolver extension.
+- **Memory (0 tools, 0 plugins):** agent appends dated lines to `.lich/memory/MEMORY.md` via `write_file`/`edit_file`; 200-line cap in the tiny helper. No system-prompt injection.
+- **Milestone (1):** the M4 e2e ("add `hash_text` to yourself" → read docs → write tool → tests green → gated commit) as the single release's acceptance test, plus a self-improvement guide in `docs/` — the guide *is* the first skill.

package/docs/design/self-improvement-loop.md

@@ -0,0 +1,166 @@
+# Lich Self-Improvement Loop — Design Draft v2 (FOR REVIEW)
+
+Status: DRAFT v2 — revised after council round 1. Round 1 verdicts: security
+REJECT, simplicity APPROVE-WITH-CHANGES, architecture APPROVE-WITH-CHANGES.
+Traceability: [security](council/security-review.md),
+[simplicity](council/simplicity-review.md),
+[architecture](council/architecture-review.md).
+
+## What changed from v1 (and why)
+
+| v1 element | Round-1 verdict | Resolution in v2 |
+| --- | --- | --- |
+| Skills toolset (save/search/read) | CUT — rebadged docs tools | Skills = `.md` files written with existing `write_file`, found by `docs_search` with a configurable docs root (resolver seam already exists) |
+| LLM consolidation pass (reflector) | Cut (simplicity) + unschema'd LLM writes (security) | Cut entirely. The agent appends its own dated lesson lines to `MEMORY.md` when a run teaches something |
+| Memory in system prompt | Persistent prompt injection (security) | Cut. `MEMORY.md` is never auto-loaded; it is read via existing tools when relevant |
+| `repo_status` | Cut (simplicity) | Cut — documented `terminal git status` recipe |
+| `git_commit` tool + gatekeeper | Kept by all three, for different reasons | Kept as the named choke point; veto mechanism redesigned (see below) |
+| "tests ran in same run" seal | TOCTOU-forgable (security) | Replaced with a HEAD-based seal: gatekeeper verifies the committed tree hash equals the tree that passed tests, captured at test time |
+| 10-min run_tests cap | Unreachable — executor caps at 30s (architecture) | Per-tool `timeout_ms` added to `Tool` interface; executor honors `tool.timeout_ms ?? 30000` |
+| M4 same-run self-registration | Impossible (bun module cache) | Two-process demo: run 1 implements+tests+commits; fresh process verifies the new tool loads |
+
+## Scope (final)
+
+Two new builtin tools, one core plugin, zero new LLM pipelines.
+
+### 1. `run_tests` tool
+
+- Args: `{filter?: string}` (vitest file filter), no path arg — pinned to the
+  lich repo (Q1: own repo only in v1).
+- Executes `test_command` from config (default
+  `node node_modules/vitest/vitest.mjs run`) with cwd = lich repo root
+  (`import.meta.url`-relative resolution), `tool timeout_ms: 600000`.
+- Structured result: `{ok, output: "PASSED 205/205" | failing test names
+  (max 10), error?}`. Output clamped to 2000 chars.
+- Concurrency guard: module-level mutex — a second concurrent `run_tests`
+  returns `{ok: false, error: "run_tests_busy"}` (NFS + vitest do not
+  tolerate two pools on one repo).
+
+### 2. `git_commit` tool
+
+- Args: `{message: string, paths: string[] (1..50, relative to work_dir)}`.
+- Behavior: verify repo (git dir exists); refuse if HEAD is unreachable;
+  `git add -- <paths>` exactly (no `-A` ever); verify index non-empty;
+  refuse when paths resolve outside work_dir (resolve_safe_path) or match
+  secret-ish basenames (`.env`, `*.pem`, `id_rsa*`); commit; return new
+  short SHA. Never pushes.
+- `timeout_ms: 60000`.
+
+### 3. Gatekeeper plugin (core, `src/plugins/builtin/gatekeeper.plugin.ts`)
+
+Fail-closed by construction (the security hole in v1 was fail-open):
+
+- `git_commit` is registered ONLY if the gatekeeper plugin loaded
+  successfully. No gatekeeper → no commit tool at all. The agent cannot
+  "fix" this without editing code that only a human commits (push is
+  human-only, and a fresh checkout restores it).
+- Veto rule: a `git_commit` call is allowed only when the plugin's own
+  run-scoped state records a `run_tests` success whose captured
+  `HEAD + index-hash` matches the tree about to be committed (closes the
+  TOCTOU: tests → edit → commit no longer passes, because edit changes the
+  index hash). Re-running `run_tests` after edits is the honest path.
+- `max_commits_per_run: 1` (default, config-tunable) — enforced from
+  run-scoped state.
+- `allow_self_commit: false` default — commit tool registered but always
+  vetoed until the human sets it true in config (explicit opt-in).
+- **Honest floor (documented limitation, not solved in v1):** the `terminal`
+  tool can still run raw `git commit`. Mitigations: gatekeeper also vetoes
+  `terminal` calls whose command matches `/git\s+(commit|push|remote)/`
+  (denylist, best-effort — documented as bypassable by a determined agent),
+  and the real boundary is human review of the local repo + push.
+  This is stated in the docs as a limitation, not hidden.
+
+### 4. Run-scoped plugin state (the missing channel)
+
+- `plugins/types.ts`: `HookContext` gains `run_state: Map<string, unknown>`
+  and `work_dir` becomes guaranteed (Agent always constructs it).
+- `agent/loop.ts`: `run_tool_calls` passes a per-run `ToolContext` (built
+  once by Agent.run, carrying `work_dir` + `env` from config) into
+  `deps.tools.execute` — today no context is passed and hooks see
+  `process.cwd()` (architecture finding #2).
+- Gatekeeper keeps its state in `run_state` (per-run map instance), so
+  resets are automatic per run and there is no cross-run module state.
+
+### 5. Skills and memory — conventions, not machinery
+
+- Skills: `.md` files under `.lich/skills/` written by the agent with
+  `write_file`; found with `docs_search` once its root resolution gains an
+  ordered multi-root list (config `docs_roots`, default
+  `[".lich/skills", "docs"]`). Cache: invalidated per call batch keyed by
+  root (replaces the single-slot memoization).
+- Memory: `MEMORY.md` in the lich repo (human-reviewable, committed),
+  agent appends dated lines via `write_file`/`edit_file` when a run
+  warrants it. No auto-load. No LLM curation.
+
+### 6. The demo (acceptance test)
+
+Two-process e2e with a mock provider and a temp git repo under `test/.tmp/`:
+
+- **Process A**: prompt "add a `hash_text` tool to yourself, following
+  `docs/architecture/extending.md`" → agent reads docs, writes the tool
+  file + fixture, registers it in `builtin/index.ts` (usable only after
+  restart — module cache, stated honestly), calls `run_tests` (mock
+  subprocess runner injectable in tests), calls `git_commit` (gatekeeper
+  verifies seal). Also the negative path: attempt commit BEFORE
+  `run_tests` → vetoed; commit after editing without re-testing → vetoed
+  (index hash mismatch).
+- **Process B**: fresh agent instance (temp repo work_dir) — `hash_text`
+  tool is registered and functional; `git log` shows exactly 1 commit.
+- Human reviews the commit in the real repo (push stays human-only).
+
+## Non-goals (unchanged from v1, plus round-1 additions)
+
+- No push tool. No cron/triggered self-modification. No auto-merge.
+- No consolidation/reflector LLM pipeline in v1 (revisit only if manual
+  MEMORY.md curation proves too tedious in practice).
+- No skills auto-loading into context (Q2: explicit search only).
+- No skill versioning frontmatter (Q5: YAGNI).
+- No terminal content inspection beyond the git-command denylist (v1 is
+  honest about the floor: human review is the boundary).
+
+## Config additions (zod, all optional with defaults)
+
+- `test_command: string` (default as above)
+- `allow_self_commit: boolean` (default false)
+- `max_commits_per_run: number` (default 1, min 1, max 10)
+- `docs_roots: string[]` (default `[".lich/skills", "docs"]`) — also makes
+  plugin-bundled docs work (answering the original plugin-docs question)
+- Consolidation provider slot: not needed (consolidation cut)
+
+## Test strategy (architecture corrections applied)
+
+- `run_tests`: injectable subprocess runner; timeout override honored
+  (proves the 30s default no longer kills it); busy-mutex path.
+- `git_commit`: temp git repo with `GIT_CONFIG_GLOBAL=/dev/null` +
+  `GIT_CONFIG_NOSYSTEM=1` env isolation; seal-verification veto paths.
+- Gatekeeper: fail-closed registration (gatekeeper load failure → no
+  `git_commit` in registry); per-run state reset across two runs in one
+  process.
+- e2e two-process demo as above (no network, mock provider).
+- Known NFS flakes: suite-level afterAll cleanup timeouts are retried
+  standalone per existing precedent; run_tests tool itself must tolerate
+  the 170s+ suite duration.
+
+## Milestone plan (collapsed per simplicity verdict)
+
+Single release **v0.4.0**, one milestone:
+
+1. Per-tool `timeout_ms` + per-run ToolContext plumbing (loop + agent +
+   executor).
+2. `run_tests` + `git_commit` tools (+ tests).
+3. Gatekeeper plugin with fail-closed registration + seal + caps (+ tests).
+4. Docs-roots parameterization (skills become findable; plugin docs ride
+   along).
+5. Docs (user-guide self-improvement page + architecture page update) and
+   the two-process e2e demo.
+
+## Open questions (v2)
+
+- Q6 (was Q3, unresolved): default `max_commits_per_run = 1` — acceptable?
+- Q7: Should the gatekeeper's terminal-git denylist be config-extensible
+  (`deny_command_patterns: string[]`) or hardcoded for v1? (Simplicity
+  says hardcoded; security says config risks foot-guns. Default proposal:
+  hardcoded v1, revisit on demand.)
+- Q8: Should `run_tests` auto-run before `git_commit` when the seal is
+  stale (agent convenience) or always require the explicit call? Default
+  proposal: explicit call only — transparency over convenience.

package/docs/getting-started.md

@@ -0,0 +1,133 @@
+# Getting started
+
+> What you'll learn: how to install Lich, configure a provider three different ways, and get your first reply through the one-shot CLI, the TUI, and the gateway webhook.
+
+## Prerequisites
+
+- Node >= 20 or Bun (Bun recommended for development; both run the same code).
+- A model endpoint: a local [Ollama](https://ollama.com) server, an OpenAI or Anthropic api key, or any OpenAI-compatible API (OpenRouter, vLLM, LM Studio, ...).
+- A clone of the Lich repository (for `bun src/cli.ts ...` commands) or an installed `lich` binary. The examples below use `bun src/cli.ts`; substitute `lich` if you installed the package.
+
+```sh
+git clone <your-fork>/lich && cd lich
+bun install
+bun src/cli.ts --version   # -> 0.2.0
+```
+
+## Choose a configuration path
+
+Lich needs exactly one thing before it runs: a model. You can provide it three ways, and they can be mixed (flags override env vars, and both override the config file).
+
+### Path A: environment variables only
+
+```sh
+# ollama — no api key needed
+LICH_PROVIDER_KIND=ollama LICH_MODEL=llama3.2 bun src/cli.ts "Reply with ok"
+
+# openai-compatible (api.openai.com/v1 by default)
+LICH_PROVIDER_KIND=openai_compat LICH_MODEL=gpt-4.1-mini bun src/cli.ts "Reply with ok"
+
+# anthropic
+LICH_PROVIDER_KIND=anthropic LICH_MODEL=claude-sonnet-4 bun src/cli.ts "Reply with ok"
+```
+
+Defaults per kind when `LICH_BASE_URL`/`LICH_API_KEY_ENV` are unset: `openai_compat` uses `https://api.openai.com/v1` and reads `OPENAI_API_KEY`; `anthropic` uses `https://api.anthropic.com` and reads `ANTHROPIC_API_KEY`; `ollama` uses `http://localhost:11434` and needs no key.
+
+### Path B: the `config` template (recommended)
+
+```sh
+mkdir -p .lich
+bun src/cli.ts config > .lich/config.json
+# edit .lich/config.json and replace "<model-name>"
+```
+
+`lich config` honors `LICH_PROVIDER_KIND` and `LICH_MODEL` when you have them set, and otherwise prints an ollama-oriented template. The file is picked up automatically from `.lich/config.json` in the working directory (or `~/.config/lich/config.json` as a fallback) — after this, plain `bun src/cli.ts "task"` needs no env vars.
+
+### Path C: an explicit config file
+
+```sh
+bun src/cli.ts --config ./lich.json "Reply with ok"
+```
+
+The full schema is documented in [the CLI reference](user-guide/cli.md#config-file-reference). Search order: `--config` path first (must exist), then `./.lich/config.json`, then `~/.config/lich/config.json`.
+
+## Your first one-shot
+
+```sh
+bun src/cli.ts "Use the list_dir tool to list the current directory then reply done"
+```
+
+Observed output (stderr progress, then the final answer on stdout):
+
+```
+[lich] turn 1
+[lich]   list_dir: ok
+
+done
+```
+
+Exit code `0` means the model produced a final answer; `1` means the turn budget ran out or the run failed (see [exit codes](user-guide/cli.md#exit-codes)).
+
+## Your first TUI session
+
+```sh
+bun src/cli.ts tui
+```
+
+Type a message and press Enter. The transcript shows your line, live tool-call rows while the agent works, and the reply; the status bar at the bottom tracks turns, tokens, and the session file path. Slash commands: `/help`, `/model`, `/usage`, `/clear`, `/sessions`, `/exit`. Details in [the TUI guide](user-guide/tui.md).
+
+## Your first gateway webhook
+
+```sh
+bun src/cli.ts gateway webhook
+```
+
+In another terminal:
+
+```sh
+curl -s -X POST http://localhost:8089/message \
+  -H "content-type: application/json" -d '{"text": "hello"}'
+```
+
+Observed response shape (the `reply` text is the model's answer; `usage` is `null` on this endpoint):
+
+```json
+{"reply":"Hello! How can I help you today? I can assist with coding, file management, running commands, web searches, and more — just let me know what you'd like to do.","usage":null}
+```
+
+Stop the gateway with Ctrl+C (SIGINT) or `kill` (SIGTERM); both shut down adapters cleanly.
+
+## Where sessions live
+
+Every run appends a JSONL transcript to `.lich/sessions/` (override with `--session-dir` or `session_dir` in config). File names encode time, a counter, and the origin label — `...-tui.jsonl` for TUI runs, `...-gw-webhook-<chat>.jsonl` for gateway conversations.
+
+Each line is a JSON record with a `ts`, a `kind` of `message` or `meta`, and the payload:
+
+```json
+{"ts":"2026-09-13T05:21:25.778Z","kind":"message","message":{"role":"user","content":"Use the list_dir tool to list the current directory then reply done"}}
+{"ts":"2026-09-13T05:21:25.852Z","kind":"message","message":{"role":"assistant","content":"","tool_calls":[{"id":"ollama_mtzd9c8b_1","name":"list_dir","args":{}}]}}
+```
+
+Print just the conversation with `jq`:
+
+```sh
+jq -r 'select(.kind=="message") | "\(.message.role): \(.message.content)"' .lich/sessions/<file>.jsonl
+```
+
+## Troubleshooting
+
+| Symptom | Cause and fix |
+| --- | --- |
+| `no model configured: set LICH_MODEL, pass --model, or create .lich/config.json` | No provider was resolvable. Set `LICH_MODEL`, pass `--model`, or save a config file (`bun src/cli.ts config`). |
+| `lich: config not found: <path>` | `--config` was given a path that does not exist. Check the path or drop the flag to use discovery. |
+| Provider error `kind=auth`, http 401/403 | The api key is missing or wrong. Verify the env var named by `LICH_API_KEY_ENV` (default `OPENAI_API_KEY`/`ANTHROPIC_API_KEY`) is exported in the same shell. |
+| `fetch failed` / connection refused | The endpoint is unreachable. For ollama, check `ollama serve` is running on `http://localhost:11434`; for remote APIs, check `LICH_BASE_URL`. |
+| `[lich] budget exhausted after N turns` | The task did not finish within `max_turns` (default 25). Raise it with `--max-turns 50` or in config. |
+| `unknown flag: --foo` | Flag typo, or the flag was placed where a subcommand is expected. Run `bun src/cli.ts --help`. |
+
+## Next steps
+
+- All four CLI modes, flags, and provider resolution: [CLI reference](user-guide/cli.md).
+- Slash commands and the status bar: [TUI guide](user-guide/tui.md).
+- Telegram, Discord, Twitch, and webhook setup: [Gateway guide](user-guide/gateway.md).
+- Embedding the agent in your own TypeScript: [Library guide](user-guide/library.md).