pi-jev-wiki 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+## 0.2.0 — 2026-09-19
+
+First public release. A pi package that builds and maintains a project mental-model wiki,
+with [Jev](https://typesafe.ai) (TypeSafe System One) as the calibrated decision layer.
+
+### Intake
+
+- Research ingest (`wiki_ingest`): immutable raw sources, claim extraction with verbatim quote
+  validation, Jev adjudication, and a placement brief.
+- Agent insight capture (`wiki_insights`): agent-authored insight lists with evidence pointers,
+  Jev-filtered and placed; sessions stored as ordinary raw sources.
+- `derivable_from_code` gate: implementation detail visible in the repository is rejected rather
+  than duplicated in the wiki.
+- Trust tiers: `verified_in_repo`, `source_document`, `user_stated`, `inference`, `speculation`.
+- Automated capture on `agent_settled` and `session_before_compact`, with a Jev pre-screen,
+  turn-count minimum, debounce, and in-process guard; pending-capture handoff when no follow-up
+  turn can run. Recurrence promotion from the session log.
+
+### Maintenance
+
+- `wiki_sync`: file-linked claims re-verified against commits since the last baseline; verdicts
+  `no_impact`, `needs_recheck`, `supersede`, `contradict`, with review queueing.
+- Two-way invalidation at ingest (new supersedes old, old supersedes new, conflicts).
+- Corroboration counting on reinforcement and explicit supersession records.
+- `wiki_review`: agent-managed queue with criticality-gated user escalation.
+- `wiki_lint`: TOC reconciliation, broken links, age-gated orphans, raw backlog, unbacked claims,
+  Jev contradiction checks, and duplicate/consolidation candidates.
+- `wiki_remove` for retiring obsolete pages.
+
+### Writing
+
+- Guided (default), draft, and auto writer modes; autonomy downgrades as claim criticality rises.
+- Nested-LLM writer returns prose only; code assembles frontmatter from adjudication results.
+- Writer grounding check flags auto-written numbers/URLs absent from evidence.
+
+### Retrieval and scale
+
+- `wiki_toc`: compact agent-facing TOC above 60 pages, per-topic tables, complete `index.md`.
+- Search engines: `index`, in-process BM25 with cache invalidation, and a qmd CLI adapter.
+- Paged-choice tournament for placement beyond 255 candidates.
+- Verified at 1,000 pages: BM25 first search ~170 ms, deterministic lint ~2.6 s.
+
+### Safety and operations
+
+- Cross-process wiki lock with stale takeover; shared-state mutations are atomic across sessions.
+- Best-effort secret/PII redaction before content is written or sent to a model.
+- `wiki_doctor`: config, endpoint, key, `.env` gitignore, layout, lock, ledger, queue, sync, search.
+- `wiki_setup`: key status/guide/write-env/test for TypeSafe and OpenRouter; never echoes the key.
+- Decision ledger (`.jev-wiki/decisions.jsonl`) recording agent, Jev, and code decisions.
+- Offline unit tests with a fake Jev client, plus smoke, paging, and scale test scripts.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 jev-wiki contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,152 @@
+# pi-jev-wiki
+
+[![npm version](https://img.shields.io/npm/v/pi-jev-wiki.svg)](https://www.npmjs.com/package/pi-jev-wiki)
+[![license](https://img.shields.io/npm/l/pi-jev-wiki.svg)](LICENSE)
+
+A pi package that gives coding agents a maintained **mental model of a project**: module
+responsibilities, boundaries, data flow, invariants, decisions, and change impact — stored as a
+markdown wiki and maintained with **Jev** (TypeSafe's calibrated decision model).
+
+The npm package is **`pi-jev-wiki`** (matching the repository); the extension, tools, and config
+files keep the `jev-wiki` name.
+
+Jev decides what is grounded, derivable, durable, and where it belongs. The agent writes. Code
+owns every threshold.
+
+- Read [`PLAN.md`](docs/plans/PLAN.md) for the design and phases.
+- Read [`CRITIQUE.md`](docs/CRITIQUE.md) for the risk analysis and efficiency evaluation.
+- Research sources live in [`research/`](research/README.md).
+
+## What it does
+
+| Tool | Purpose |
+|---|---|
+| `wiki_toc` | the wiki table of contents (compact above 60 pages, per-topic tables) |
+| `wiki_ask` | find pages and excerpts (index / BM25 / qmd engine) |
+| `wiki_ingest` | ingest a document: raw source → claims → Jev verdicts → placement brief |
+| `wiki_insights` | capture agent insights, Jev-filtered and placed |
+| `wiki_finalize` | update TOC/log after writing pages, check links |
+| `wiki_sync` | re-verify file-linked claims against commits since the last sync |
+| `wiki_review` | list or resolve review items; critical items escalate to the user |
+| `wiki_lint` | health checks: TOC, links, orphans, unbacked claims, contradictions, duplicates |
+| `wiki_remove` | delete obsolete pages and their TOC entries |
+| `wiki_structure` | deterministic module/dependency map and architecture coverage |
+| `wiki_doctor` | config, key, lock, ledger, queue, git/sync, search health |
+| `wiki_setup` | inspect or configure the Jev API key (TypeSafe or OpenRouter) |
+| `wiki_status` | pages, raw sources, ledger, consultations, Jev usage |
+| `/wiki:ingest`, `/wiki:capture`, `/wiki:sync`, `/wiki:review`, `/wiki:lint`, `/wiki:status` | user-facing commands |
+
+The wiki is **never injected** into sessions. The table of contents is available like a skill
+(`wiki_toc` + the `llm-wiki` skill); the agent consults it on demand.
+
+## Install
+
+```bash
+pi install /path/to/jev-wiki        # local folder
+pi install npm:pi-jev-wiki             # once published
+pi install git:github.com/xAndreiLi/pi-jev-wiki@v0.2.0
+```
+
+For development, load it directly:
+
+```bash
+pi -e ./src/extension.ts --skill ./skills/llm-wiki
+```
+
+## API keys
+
+Both supported providers speak the same Jev Decisions schema:
+
+| Provider | Env var | Endpoint | Context | Notes |
+|---|---|---|---|---|
+| **TypeSafe** (official, default) | `TYPESAFE_API_KEY` (or `JEV_TOKEN`) | `https://api.typesafe.ai/v1/systemone` | 64k | $0.042/Mtok input, output free |
+| **OpenRouter** | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/alpha/decisions` | 32k advertised | set `provider: "openrouter"`; or reuse pi's `/login openrouter` credential |
+
+Put the key in a project-root `.env` (gitignored — `wiki_doctor` verifies):
+
+```bash
+TYPESAFE_API_KEY=...     # TypeSafe
+# or
+OPENROUTER_API_KEY=...   # OpenRouter
+```
+
+Or reference it from config with `$VAR` indirection:
+
+```json
+{ "provider": "openrouter", "apiKey": "$OPENROUTER_API_KEY", "model": "~typesafe/jev-latest" }
+```
+
+When a key is missing, ask the agent to run `wiki_setup`:
+
+- `action=status` — provider, endpoint, and where the key came from (never the value)
+- `action=guide provider=typesafe|openrouter` — exact env var, file, and config steps
+- `action=write-env provider=... apiKey=...` — writes to `.env` after checking gitignore
+- `action=test` — one tiny live call to verify connectivity and auth
+
+## Configuration
+
+Optional overrides in `~/.pi/agent/jev-wiki.json` or project `.pi/jev-wiki.json`
+(see `src/config.ts` for all keys):
+
+```json
+{
+  "provider": "typesafe",
+  "model": "jev-latest",
+  "wikiRoot": "docs/wiki",
+  "writer": { "mode": "guided" },
+  "thresholds": { "autoAccept": 0.8, "minDerivable": 0.5 }
+}
+```
+
+## Wiki layout
+
+```
+docs/wiki/
+├── raw/<topic>/YYYY-MM-DD-slug.md   # immutable sources (documents + session captures)
+└── wiki/
+    ├── index.md                     # generated table of contents
+    ├── log.md                       # generated append-only log
+    ├── architecture/                # module-*, flow-*, layer-*
+    ├── invariants/                  # invariant-*
+    ├── decisions/                   # decision-* (ADR-style)
+    ├── impact/                      # impact-* (derived)
+    └── <topic>/                     # gotcha-*, glossary-*, concept-*, summary-*
+```
+
+Runtime state lives in `docs/wiki/.jev-wiki/` (gitignored): the decision ledger
+(`decisions.jsonl`), raw-source hash index, and session log.
+
+## Development
+
+```bash
+npm install
+npx tsc --noEmit      # typecheck
+npm run smoke         # deterministic checks + live Jev round-trips
+```
+
+## Repository layout
+
+```text
+src/          pi extension, Jev client, pipelines, wiki primitives
+skills/       llm-wiki skill + page templates (the schema layer)
+scripts/      unit, smoke, paging, and scale tests
+docs/
+  plans/      PLAN.md (master plan) + plans index
+  DESIGN.md   detailed technical design
+  CRITIQUE.md pre-implementation critique and efficiency evaluation
+  HARDENING.md hardening roadmap with statuses
+  notes/      source notes used for dogfooding
+  wiki/       this project's own knowledge wiki (dogfood)
+  RELEASING.md release runbook
+research/     source material gathered during design
+```
+
+The published npm package ships only `src/`, `skills/`, `README.md`, and `LICENSE`; everything
+under `docs/` and `research/` stays in the repository.
+
+## Status
+
+P0 (walking skeleton) implemented: both intake channels (research ingest + agent insights),
+architecture-first pages, TOC/log, decision ledger, Jev client with retries and usage accounting,
+guided writing. See `docs/plans/PLAN.md` §9 for P1–P3 (change-driven invalidation, paged routing beyond 250
+pages, agent-managed review queue, lint, decision-quality evaluation).

package/package.json

@@ -0,0 +1,63 @@
+{
+  "name": "pi-jev-wiki",
+  "version": "0.2.0",
+  "description": "Project mental-model wiki for pi, maintained by agent insights and Jev decisions",
+  "type": "module",
+  "license": "MIT",
+  "author": "xAndreiLi",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/xAndreiLi/pi-jev-wiki.git"
+  },
+  "homepage": "https://github.com/xAndreiLi/pi-jev-wiki#readme",
+  "bugs": {
+    "url": "https://github.com/xAndreiLi/pi-jev-wiki/issues"
+  },
+  "engines": {
+    "node": ">=22"
+  },
+  "files": [
+    "src",
+    "skills",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "keywords": [
+    "pi-package",
+    "pi",
+    "wiki",
+    "jev",
+    "knowledge-base"
+  ],
+  "pi": {
+    "extensions": [
+      "./src/extension.ts"
+    ],
+    "skills": [
+      "./skills"
+    ]
+  },
+  "scripts": {
+    "typecheck": "tsc --noEmit",
+    "smoke": "jiti scripts/smoke.ts",
+    "test:paging": "jiti scripts/paging-test.ts",
+    "test:scale": "jiti scripts/scale-test.ts",
+    "test:unit": "jiti scripts/unit-test.ts",
+    "test:all": "tsc --noEmit && jiti scripts/unit-test.ts && jiti scripts/scale-test.ts",
+    "prepublishOnly": "npm run typecheck && npm run test:unit"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-ai": "*",
+    "@earendil-works/pi-coding-agent": "*",
+    "typebox": "*"
+  },
+  "devDependencies": {
+    "@earendil-works/pi-ai": "0.85.1",
+    "@earendil-works/pi-coding-agent": "0.85.1",
+    "@types/node": "^26.6.2",
+    "jiti": "^2.7.0",
+    "typebox": "^1.3.34",
+    "typescript": "^5.5.4"
+  }
+}

package/skills/llm-wiki/SKILL.md

@@ -0,0 +1,143 @@
+---
+name: llm-wiki
+description: "Use when working with the project knowledge wiki: consulting architecture/decisions before changes, ingesting documents, capturing session insights, or maintaining the wiki. Triggers: 'wiki', 'what do we know about', 'why is it built this way', 'ingest', 'capture insights', 'add to wiki'."
+---
+
+# Project Wiki (jev-wiki)
+
+The project has a knowledge wiki at `docs/wiki/` (configurable) holding the **mental model** of
+the system: how it is structured, why, and what a change touches. It is not a copy of the code.
+Jev (a calibrated decision model) judges whether claims are grounded, derivable, durable, and
+where they belong; you do the writing.
+
+## Setup (when no key is configured)
+
+If Jev calls fail with an authentication error, or the user asks how to connect a provider:
+
+1. `wiki_setup action=status` — shows the provider, endpoint, and where the key came from (never the value).
+2. `wiki_setup action=guide provider=typesafe|openrouter` — exact env var, file, and config steps.
+3. Ask the user for the key, then either have them add it to the project `.env` or write it with
+   `wiki_setup action=write-env provider=... apiKey=...` (this checks that `.env` is gitignored first).
+4. `wiki_setup action=test` — one tiny live call to verify connectivity and auth.
+
+Never print the key value. TypeSafe uses `TYPESAFE_API_KEY` (`JEV_TOKEN` also works); OpenRouter
+uses `OPENROUTER_API_KEY` with provider `openrouter`, or pi's own `/login openrouter` credential.
+
+## The quality bar
+
+File only what the repository cannot answer cheaply:
+
+- **Delete test** — if deleting the page would just send a future agent back to reading code, it
+  does not belong.
+- **Decision test** — does it help decide *where a change belongs* or *what a change breaks*?
+
+Good: module responsibilities and boundaries, dependency direction, data flow, invariants,
+decisions with rationale, change-impact knowledge, historical attempts, external constraints,
+domain glossary. Bad: function bodies, obvious implementation, transient task state, code excerpts.
+
+## Layout
+
+```
+docs/wiki/
+├── raw/<topic>/YYYY-MM-DD-slug.md   # immutable sources (never edit)
+└── wiki/
+    ├── index.md                     # TOC — generated, never hand-edit
+    ├── log.md                       # append-only, generated
+    ├── architecture/                # module-*.md · flow-*.md · layer-*.md
+    ├── invariants/                  # invariant-*.md
+    ├── decisions/                   # decision-*.md (ADR-style)
+    ├── impact/                      # impact-*.md (derived)
+    └── <topic>/                     # gotcha-*.md · glossary-*.md · concept-*.md · summary-*.md
+```
+
+## When to consult
+
+Before architectural, cross-cutting, or unfamiliar changes; when planning work; when a project
+term is unclear; before answering "how does X work here?".
+
+1. `wiki_toc` — browse the table of contents (filter by topic/tag/query).
+2. `wiki_ask` — find relevant pages for a question.
+3. `read` the pages it returns, then cite them (`docs/wiki/wiki/...`) in your answer.
+4. Never invent wiki content; if it is not there, say so and consider whether it should be.
+
+## Operating the wiki
+
+### Respect the verdicts
+
+Jev's adjudication is binding: write only claims the brief marks **File** or **Reinforce**.
+Rejected claims are **not** written to the wiki — even when the user explicitly asks for that
+content. Report what was rejected and why, so the user can supply better evidence or a stronger
+artifact. Never hand-write a rejected claim into a page.
+
+### Ingest a document (research channel)
+
+1. `wiki_ingest` with `path` (or `text`) — it stores the raw source, extracts claims, and returns
+   a Jev-verified brief.
+2. Write or merge the **accepted** pages (guided mode: you write).
+3. `wiki_finalize` with every touched page.
+
+### Capture session insights (work channel)
+
+1. Compose atomic insights with evidence pointers (files, commits, tests, user statements).
+   No transient state, no code snippets, nothing derivable from the repo.
+2. `wiki_insights` with the list — Jev filters (derivable/durable/sensitive), relates them to
+   existing knowledge, and chooses placement. Include file/commit evidence: file evidence is read
+   and excerpted for Jev, which grounds the decision.
+3. Write or merge the **accepted** pages, then `wiki_finalize`.
+
+## Page format
+
+```markdown
+---
+title: Auth module
+type: architecture/module
+topic: architecture
+summary: Owns token validation and session issuance.
+tags: [auth, security]
+updated: 2026-09-19
+sources: [raw/auth/2026-09-19-auth-notes.md]
+claims:
+  - id: c1
+    text: "Token validation lives in the auth module and is the only issuer of sessions."
+    status: verified        # verified | user-stated | unsupported | contradicted | disputed | superseded
+    support: 0.96
+    evidence: ["raw/auth/2026-09-19-auth-notes.md"]
+files: [src/auth/index.ts]
+---
+
+# Auth module
+
+Responsibility, public surface, dependencies, invariants, and key files.
+
+## Invariants
+- ...
+
+## Change impact
+- ...
+
+## See also
+- [Session flow](../architecture/flow-session.md)
+```
+
+Rules:
+
+- Every load-bearing claim points at raw evidence or a file/commit/test; numbers and quotes must
+  exist verbatim in the source.
+- Never silently rewrite history. Superseded or contradicted claims keep their text, get
+  `status: superseded|disputed`, and link the newer claim/source.
+- Relative links inside the wiki; project-relative paths when citing in conversation.
+- One level of topic subdirectories; page names are kebab-case and describe the subject, not the
+  source file.
+- Prefer updating an existing page over creating a new one — but if nothing fits, create one and
+  let the TOC absorb it.
+
+## Maintenance
+
+- If a page's claim is invalidated by code you changed this session, update it (or mark it
+  `needs_recheck`) and mention it in your summary.
+- When you notice a contradiction between pages, mark both with `status: disputed` and cross-link
+  them; do not silently pick a winner.
+- Report broken links and orphans to the user; `wiki_finalize` checks links on touched pages.
+
+Templates live in `references/` next to this skill: `module.md`, `flow.md`, `invariant.md`,
+`decision.md`, `gotcha.md`.

package/skills/llm-wiki/references/decision.md

@@ -0,0 +1,32 @@
+---
+name: decision-page
+description: Template for ADR-style decision records.
+---
+
+# <Decision title>
+
+**Status.** accepted | superseded by [link] | deprecated
+
+**Date.** YYYY-MM-DD
+
+## Context
+
+- The forces at play: constraints, requirements, prior art, deadlines.
+
+## Options considered
+
+1. Option A — pros, cons.
+2. Option B — pros, cons.
+
+## Decision
+
+- What was chosen and the decisive reasons.
+
+## Consequences
+
+- What becomes easy, what becomes hard, what must now be maintained.
+- What would trigger revisiting this decision.
+
+## Evidence
+
+- Sources, discussions, experiments, or commits that support the rationale.

package/skills/llm-wiki/references/flow.md

@@ -0,0 +1,32 @@
+---
+name: flow-page
+description: Template for end-to-end data or control flow wiki pages.
+---
+
+# <Flow name>
+
+**Trigger.** What starts this flow.
+
+**Participants.** The modules/services involved, in order.
+
+## Steps
+
+1. Entry point (`src/...`) — input shape.
+2. Transformation / hand-off — what changes and where.
+3. Exit — output shape, side effects, storage.
+
+## Invariants
+
+- Ordering guarantees, idempotency, exactly-once/at-least-once properties.
+
+## Failure modes
+
+- Where it can stall, retry, or duplicate; how errors surface.
+
+## Change impact
+
+- Steps and modules affected by changes to each stage.
+
+## See also
+
+- Related module, invariant, and decision pages.

package/skills/llm-wiki/references/gotcha.md

@@ -0,0 +1,16 @@
+---
+name: gotcha-page
+description: Template for footguns and non-obvious failure modes.
+---
+
+# <Gotcha>
+
+**Symptom.** What a developer observes when they hit this.
+
+**Cause.** The non-obvious mechanism behind it.
+
+**Avoidance.** What to do instead; the safe pattern.
+
+**Detection.** How to tell you are about to hit it (tests, lints, logs).
+
+**Related.** Invariants, decisions, or modules involved.

package/skills/llm-wiki/references/invariant.md

@@ -0,0 +1,20 @@
+---
+name: invariant-page
+description: Template for rules that must always hold.
+---
+
+# <Invariant statement>
+
+**Statement.** One sentence, testable, ideally in "must / must never" form.
+
+**Why it exists.** The failure it prevents; the decision that created it.
+
+**Where it is enforced.** Code locations, tests, checks (pointers only).
+
+**What breaks if violated.** Symptoms and blast radius.
+
+**How to verify.** The test or command that proves it holds.
+
+## Related
+
+- Decisions, modules, and gotchas that interact with this invariant.

package/skills/llm-wiki/references/module.md

@@ -0,0 +1,30 @@
+---
+name: architecture-page
+description: Template for architecture module/flow/layer wiki pages.
+---
+
+# <Subject>
+
+**Responsibility.** What this module/flow/layer owns, in one or two sentences.
+
+**Public surface.** The contracts other parts of the system rely on.
+
+**Dependencies.** What it depends on, and the direction (never the reverse).
+
+**Key files.** `src/...` — pointers only, no code.
+
+## Invariants
+
+- Rules that must always hold; each linked to evidence in the claim frontmatter.
+
+## Failure modes
+
+- What breaks, how it surfaces, and what to do.
+
+## Change impact
+
+- If you change X, these parts are affected; tests/owners to consult.
+
+## See also
+
+- Relative links to related pages.