@event4u/agent-config 9.7.0 → 9.9.0

package/docs/contracts/local-knowledge-ingestion.md

@@ -1,96 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-23
----
-
-# Local knowledge ingestion contract
-
-**Purpose.** Freeze the input shape, bounds, storage target, and redaction defaults for the single-user, local-only knowledge surface (`/knowledge:ingest`, `/knowledge:list`, `/knowledge:forget`) **before** any implementation lands. Closes the "Copy/Paste AI" complaint from feedback A without touching OAuth, multi-tenancy, or Hard-Floor connector territory.
-
-Last refreshed: 2026-05-24. Phase 2 of the employee-product workstream.
-
-## What this doc is **not**
-
-- Not the connector contract for GitHub / Jira / Confluence — those sit behind OAuth and stay cancelled in `road-to-internal-ai-os-deployment.md` Phase 5.
-- Not a remote-fetch surface — every input must resolve to a local path on the same machine the agent runs on.
-- Not a memory replacement — ingested content lives **inside** the existing memory layer under a dedicated namespace, never as a parallel store.
-
-## Input shapes
-
-The ingestion command accepts exactly these input shapes; anything else is rejected at the input validator with a structured error and no partial write.
-
-| Input | Resolution rule | Example |
-|---|---|---|
-| `file://<absolute-path>` | Single file. Path must be absolute and inside the user's home or the project root (no `/etc`, `/var`, `~root/`, `..` escapes). | `file:///Users/maintainer/clients/acme/brief.pdf` |
-| Local folder path | Recursive walk; symlinks not followed; hidden directories skipped (`.git`, `.venv`, `node_modules`). | `/Users/maintainer/clients/acme/` |
-| `.zip` archive | Unpacked to a temp dir inside `$TMPDIR/agent-knowledge-<uuid>`, walked, then the temp dir is removed before the command returns. | `/Users/maintainer/clients/acme.zip` |
-
-**Remote URLs are rejected** — `http://`, `https://`, `s3://`, `gs://`, `azure://`. The error message names `/knowledge:ingest` as local-only by design.
-
-## Supported MIME types
-
-The ingestion module routes each file through the existing `markitdown` adapter when the MIME type is not native markdown.
-
-| MIME | Adapter | Notes |
-|---|---|---|
-| `text/markdown` | passthrough | UTF-8 only; other encodings rejected |
-| `text/plain` | passthrough | UTF-8 only |
-| `application/pdf` | `markitdown` | OCR if scanned; OCR confidence < 0.7 surfaces as `low_confidence` tag |
-| `application/vnd.openxmlformats-officedocument.wordprocessingml.document` (`.docx`) | `markitdown` | |
-| `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` (`.xlsx`) | `markitdown` | One sheet per chunk |
-| `application/epub+zip` (`.epub`) | `markitdown` | Chapter per chunk |
-| `image/png`, `image/jpeg` | `markitdown` OCR | OCR confidence stored same as PDF |
-
-Unsupported MIME → file skipped with a counted `skipped: <mime>` entry in the command summary. No partial-content writes.
-
-## Bounds (non-negotiable, enforced at command entry)
-
-| Bound | Limit | Rationale |
-|---|---|---|
-| Total ingest size | ≤ 100 MB per `/knowledge:ingest` call | Keeps a single ingestion bounded; multi-ingest is fine |
-| Document count | ≤ 1000 per call | Avoids unbounded walks on a misconfigured folder |
-| Per-file size | ≤ 20 MB | One outlier file cannot blow the budget |
-| Total memory footprint | ≤ 500 MB across all ingests | LRU eviction at the namespace level when crossed |
-| Path traversal depth | ≤ 10 directories | Cheap guard against pathological folder trees |
-
-Crossing any bound is a **hard reject** at command entry — not a warning. The command returns a structured error with the bound name and the observed value.
-
-## Storage target
-
-Ingested content lives inside the existing memory namespace as a dedicated prefix:
-
-```
-memory/
-└── knowledge/
-    ├── <ingest-id>/         # uuid7 per ingest call
-    │   ├── manifest.json    # source path, count, timestamps, redactions
-    │   └── chunks/<n>.md    # one markdown chunk per logical unit
-```
-
-- `<ingest-id>` is a uuid7 so timestamps are recoverable from the id; never user-controlled.
-- `manifest.json` is the audit row — used by `/knowledge:list` and by the LRU eviction loop.
-- Chunks are markdown only after the adapter has run; the original binary is never stored.
-
-The MCP tool `memory_retrieve` (existing surface in `agent-memory`) **must** tag retrieved entries from this namespace with `source: knowledge`. The host model decides what to do with user-supplied vs maintainer-curated entries; this contract only requires the tag.
-
-## Redaction defaults
-
-Redaction runs **before** the chunk write, never after.
-
-- **PII allowlist** — only the following identifier classes survive the ingest: project names, document titles, headings, technical terminology. Everything else that pattern-matches a PII regex set (e-mail addresses, phone numbers, IBAN, credit-card-shaped strings, SSN-shaped strings) is replaced with class placeholders (`[EMAIL]`, `[PHONE]`, `[IBAN]`, `[CC]`, `[SSN]`).
-- **Secrets never stored** — anything matching the existing `gitleaks` ruleset (or equivalent) is replaced with `[SECRET]` and the manifest's `secrets_redacted` counter is incremented. A chunk with ≥ 1 secret redaction is tagged `contains_redactions: true`.
-- **The user can opt out per-call** with `--no-redact`, but the manifest captures the flag so the audit trail names exactly which ingests bypassed redaction. The default is always redact.
-
-## Eviction policy
-
-LRU at the namespace level. When the 500 MB cap is crossed, oldest ingests (by `manifest.last_touched`) are dropped whole — never per-chunk. `last_touched` updates on every `memory_retrieve` hit against an entry in the namespace. The user can pin an ingest with `/knowledge:list --pin <ingest-id>`; pinned ingests are never evicted.
-
-## Command surface (deferred to impl PR)
-
-`/knowledge:ingest <path>`, `/knowledge:list`, `/knowledge:forget <prefix>` are defined elsewhere — this contract pins their **inputs, bounds, storage, and redaction**. The Python module that implements the file walk + MIME routing + chunk writing lives at `packages/core/installer/python/knowledge_ingest.py`, ≤ 600 LOC (bumped from the pre-impl ≤ 400 budget once five PII classes, five secret patterns, LRU eviction, manifest persistence, pin/unpin and the multi-verb CLI were all in one file — splitting would have added an import seam without changing the surface), per Phase 2 Step 2.
-
-## Open questions (Phase 2 council pass, optional)
-
-- Chunk size — fixed (e.g. 2 KB markdown) vs adaptive per document? Default: 2 KB until the recruit sessions or eval surface a reason to change.
-- OCR confidence threshold — 0.7 is a guess; first three sessions inform the right number.
-- Pinning UX — `--pin` flag vs a separate `/knowledge:pin` command. Default in this contract: a flag on `/knowledge:list`.

package/docs/contracts/local-server-api.md

@@ -1,134 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-17
----
-
-# Local server API — `agent-config ui:serve`
-
-> **Status:** active · **Stability:** beta · **Owner:** ADR-012 (TypeScript CLI shell)
-> · **Surface:** `src/server/**` · **Tests:** `tests/server/**`
-
-Locks the wire shape, security stance, and lifecycle of the embedded
-HTTP server that `agent-config ui:serve` boots. Phase 5 of
-`typescript-cli-and-local-gui-foundation` ships this contract behind
-the flipped TS binary; downstream Roadmap-2 (in-browser wizard) builds
-against it.
-
-## § 1 — Bind and port
-
-- **Bind address.** `127.0.0.1` only. The server **never** binds to
-  `0.0.0.0` or a routable interface. Enforced in
-  `src/cli/commands/uiServe.ts`.
-- **Port range.** First free TCP port in `[41000, 41999]`, scanned
-  ascending. Picker: `src/server/port.ts#pickFreePort`. If the entire
-  range is occupied, the command exits non-zero with a guidance line
-  pointing at the range.
-- **No remote access.** No published TLS, no reverse-proxy story.
-  Operators who need remote access tunnel via `ssh -L` or equivalent;
-  the package does not ship that path.
-
-## § 2 — Authentication — per-process bearer token
-
-- Minted at boot by `src/server/token.ts#mintToken` (URL-safe,
-  256 bits of entropy from `crypto.randomBytes`).
-- Required on **every** `/api/*` route via either:
-  - `Authorization: Bearer <token>`, **or**
-  - `?token=<token>` query parameter (the UI uses this so the
-    browser can bootstrap from the URL).
-- Token comparison runs through `tokensMatch` — constant-time over
-  equal-length inputs.
-- The token is printed to **stderr** by `ui:serve` once, prefixed
-  with `agent-config: token=` so log scrapers can redact it. Never
-  written to disk by the package.
-- Static UI files under `/` are **not** gated — the browser must be
-  able to fetch the HTML before it can present the token.
-
-## § 3 — Defence-in-depth headers
-
-- **Host-header guard.** `Host` MUST be `127.0.0.1:<port>` or
-  `localhost:<port>`. Other values → `421 Misdirected Request`.
-  Blocks DNS-rebinding from a malicious page that resolves an
-  attacker-controlled hostname to `127.0.0.1`.
-- **Origin allow-list.** When `Origin` is present, it MUST equal
-  `http://127.0.0.1:<port>` or `http://localhost:<port>`. Other
-  values → `403 Forbidden`. Absent header is allowed (server-to-server
-  callers, `curl` without `-H`).
-- **CORS.** No `Access-Control-Allow-Origin` header. The server is
-  same-origin to the bundled UI by construction; browsers from any
-  other origin are blocked by the same-origin policy before
-  `Origin` is even checked.
-
-## § 4 — Routes
-
-### `GET /api/v1/ping`
-
-Liveness probe. Used by:
-- The bundled UI to confirm it reached the right process.
-- `agent-config doctor-shell` (future — Phase 6).
-- CI smoke tests (`tests/cli/cli-e2e.test.ts`).
-
-**Request:** no body. Token via header or query.
-
-**Response — `200 OK`:**
-
-```json
-{
-  "ok": true,
-  "version": "2.26.0",
-  "projectRoot": "/abs/path/to/project"
-}
-```
-
-- `ok` — literal `true`.
-- `version` — value of `package.json#version` at boot time.
-- `projectRoot` — absolute path the CLI resolved (see
-  `src/cli/paths.ts`).
-
-Schema source of truth: `src/server/routes/ping.ts#PingResponseSchema`
-(zod). Test gate: `tests/server/app.test.ts`.
-
-### Future routes (Phase 6+)
-
-Reserved namespace under `/api/v1/`. Adding a new route requires:
-- A zod schema export co-located with the handler.
-- A `tests/server/` integration test that exercises the auth gate
-  alongside the happy path.
-- A CHANGELOG entry under the active Unreleased block.
-
-## § 5 — Failure modes
-
-| Condition | HTTP | Body shape |
-|---|---|---|
-| Missing/invalid token on `/api/*` | `401` | `{ "error": "Unauthorized: …" }` |
-| Disallowed `Host` | `421` | `{ "error": "Misdirected Request: …" }` |
-| Disallowed `Origin` | `403` | `{ "error": "Forbidden: …" }` |
-| Port range exhausted | (no HTTP — CLI exit non-zero) | — |
-
-## § 6 — Lifecycle
-
-- **Boot.** `ui:serve` picks a free port, mints the token, builds
-  the Fastify app via `createApp`, and prints the URL + token to
-  stderr. By default it also opens the OS browser at `/` with the
-  token in the query string; `--no-open` suppresses that step.
-- **Headless detection.** When `process.stdout.isTTY === false` AND
-  `--open` was not explicitly passed, the server boots but does not
-  attempt to spawn a browser. Useful for CI and SSH sessions.
-- **Shutdown.** SIGINT / SIGTERM stops the server cleanly; the
-  per-process token is dropped from memory.
-
-## § 7 — Stability commitments
-
-- Wire shape (route paths, response keys, status codes) — covered
-  by the contract above. Breaking change requires SemVer-major.
-- Security stance (§ 1–3) — strengthening is non-breaking;
-  weakening (e.g. removing the Host guard) is a breaking change.
-- Port range — narrowing is breaking, widening is non-breaking.
-- The token is **never** exfiltrated to logs, telemetry, or
-  observability backends by the package itself.
-
-## Related contracts
-
-- `docs/decisions/ADR-012-typescript-cli-shell.md` — the parent
-  decision (shell choice, source layout, dependency surface).
-- `docs/contracts/STABILITY.md` — what `stability: beta` means and
-  the promotion path.

package/docs/contracts/low-impact-corpus-format.md

@@ -1,95 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-13
----
-
-# `low-impact-decisions.md` — corpus format contract (step-9 P4)
-
-Parser-visible invariants for `agents/decisions/low-impact-decisions.md` and any
-upstream seed at the same path. The hardened parser lives in
-[`scripts/ai_council/low_impact_corpus.py`](../../src/scripts/ai_council/low_impact_corpus.py)
-and ships in two modes:
-
-| Mode | Entry point | Behaviour on drift |
-|---|---|---|
-| **Lenient** (routing hot-path) | `load_validated_phrases(path)` | Silently drops the offending line. Routing never blocks on a malformed corpus. |
-| **Strict** (CI lint + intake) | `parse_corpus_strict(path)` | Raises `CorpusParseError` with a typed `reason` and 1-based `line` on the first anomaly. |
-
-## Required sections
-
-```
-## On Probation
-
-<!-- intake-anchor: probation -->
-
-…
-
-## Validated
-
-<!-- intake-anchor: validated -->
-
-…
-
-## Anti-Examples (Always Ask User)
-
-…
-```
-
-- Heading level **MUST** be `##` (two hashes). `###` → `heading_drift`.
-- Trailing punctuation on a heading (`## Validated:`) → `heading_drift`.
-- Sections may appear in any order; missing sections are tolerated by
-  both modes (an empty corpus is valid).
-- The intake-anchor HTML comments **MUST** be present for the two
-  intake-bearing sections (`probation`, `validated`) once any section
-  body is present. Strict mode raises `missing_anchor` otherwise; the
-  lenient shim ignores anchors (anchors are for the intake writer, not
-  the routing reader).
-
-## Bullet shape
-
-```
-- "<phrase>" — optional trailing metadata
-```
-
-Strict invariants:
-
-| Drift | `reason` | Example |
-|---|---|---|
-| Curly opening quote (`U+201C` / `U+2018`) | `curly_quotes` | `- “foo bar”` |
-| Single-quoted phrase | `single_quotes` | `- 'foo bar'` |
-| Non-dash list marker (`*`, `+`, `1.`) | `non_dash_bullet` | `* "foo bar"` |
-| Opening `"` with no matching close on the same line | `unclosed_quote` | `- "foo bar — meta` |
-| Phrase normalises to empty (whitespace / punctuation only) | `empty_phrase` | `- "   …"` |
-
-Trailing metadata (everything after the closing `"`) is preserved on
-the `CorpusEntry.trailing_metadata` field but is **not** consulted for
-routing — only the normalised phrase is.
-
-## Phrase normalisation
-
-A phrase is normalised before equality / similarity comparison:
-
-1. Lowercase.
-2. Replace every non-`[\w\s]` character with a single space.
-3. Collapse runs of whitespace.
-4. Strip leading / trailing whitespace.
-
-This normalisation runs in both modes and is stable across the
-routing classifier (`classify_impact_with_corpus`), the intake module
-(`low_impact_intake`), and the probation gate.
-
-## Failure-mode fixtures
-
-The seven canonical failure cases ship as fixtures under
-[`tests/fixtures/corpus-robust/`](../../tests/fixtures/corpus-robust/),
-one file per `reason`. The strict-mode suite
-[`tests/test_low_impact_corpus_robustness.py`](../../tests/test_low_impact_corpus_robustness.py)
-asserts each fixture trips the matching `CorpusParseError.reason` and
-that the lenient shim degrades silently for per-bullet drift.
-
-## Cross-references
-
-- Privacy floor — `.augment/rules/low-impact-corpus-privacy-floor.md`
-- Routing — `scripts/ai_council/necessity.py § classify_impact_with_corpus`
-- Intake — `scripts/ai_council/low_impact_intake.py`
-- Promotion / pruning — `scripts/ai_council/probation_gate.py`

package/docs/contracts/mcp-beta-criteria.md

@@ -1,130 +0,0 @@
----
-stability: experimental
-mcp_scope: lite
----
-
-# MCP Beta Criteria — Promotion Gate (Hard Contract)
-
-> **Status:** Active · governs the `experimental → beta` promotion for
-> the MCP surface (`scripts/mcp_server/` local stdio kernel + the
-> hosted `internal/workers/mcp/` bridge). Owned by Phase 3 of the
-> `road-to-surface-discipline` roadmap (see `agents/roadmaps/`).
-> Companion contract:
-> [`mcp-phase-1-scope.md`](mcp-phase-1-scope.md) (local) ·
-> [`mcp-cloud-scope.md`](mcp-cloud-scope.md) (hosted).
-
-## Purpose
-
-The current MCP wording uses `experimental` across READMEs, module
-docstrings, and the initialize-result server description. There is no
-defined bar for retiring that label. This contract names six gates
-that together flip `experimental → beta`. Every gate is **observable**
-(test file, doc, or script), **falsifiable** (red is allowed; missing
-is not), and **machine-reportable** through `agent-config doctor
---check mcp-beta-readiness` (lands in Phase 3 Step 5).
-
-> **Iron Law:** all six gates must be green for the same release tag
-> before any user-visible surface drops `experimental`. A green gate
-> sheet on `main` does not authorize a back-dated wording change on a
-> release branch that did not also pass the sheet.
-
-## The six gates
-
-Each gate is owned by a single artefact. When the artefact is missing,
-Phase 3 Step 3 creates a **failing test** (`pytest.skip("pending: …",
-allow_module_level=True)` or `raise NotImplementedError("mcp-beta-gate-N
-pending")`) so the AC stays falsifiable.
-
-### Gate 1 — External-client end-to-end run
-
-At least one MCP client **outside this repo's own test harness** has
-completed a full session against MCP Lite: `initialize` →
-`prompts/list` → `prompts/get` → `resources/list` → `resources/read`
-→ shutdown. Evidence is a transcript or recorded session under
-`tests/mcp/external-clients/` plus the client name and version
-(Claude Desktop ≥ vX, Cursor ≥ vY, Zed ≥ vZ, Continue ≥ vW).
-
-### Gate 2 — Bearer-auth coverage
-
-`tests/mcp/auth/` must cover four cases against the hosted Worker
-surface — **happy path**, **401 on missing token**, **401 on expired
-token**, **401 → 200 on rotated token**. Each case asserts the wire
-envelope shape, not only the status code. Gate fails if any case is
-skipped, xfailed, or absent.
-
-### Gate 3 — Lite/Full parity smoke suite
-
-For every primitive the published surface exposes (`prompts/list`,
-`prompts/get`, `resources/list`, `resources/read`), a parametrized
-test asserts the response body from the hosted Worker (Lite) and the
-local stdio kernel (Full) **byte-identical** (modulo the documented
-deltas in `mcp-cloud-scope.md § Lite vs Full`). Failure must surface
-the diff, not just a boolean.
-
-### Gate 4 — Health endpoint under load
-
-The hosted Worker exposes `/healthz` (or equivalent) that returns a
-structured JSON envelope `{status, uptime_s, build_sha,
-last_content_refresh}`. A k6 / wrk smoke test in
-`tests/mcp/load/healthz.k6.js` proves p95 < 200 ms across 60 s at 50
-RPS. The local stdio kernel surfaces the same envelope through a
-`server/health` JSON-RPC ping.
-
-### Gate 5 — Abuse / rate-limit plan
-
-`docs/contracts/mcp-rate-limit.md` exists and pins three knobs —
-per-token RPS, per-token daily quota, per-IP burst — with a fallback
-behaviour on overrun (`429` + `Retry-After`). The Worker enforces the
-knobs; a contract test in `tests/mcp/rate-limit/` asserts that
-exceeding any knob returns `429` with a non-empty `Retry-After`.
-
-### Gate 6 — Lite ↔ Full no-drift
-
-A nightly CI job runs the Phase 3 Step 3 parity suite (Gate 3) plus a
-canary: ingest one prompt and one resource on both surfaces, hash the
-body, and assert equality. Drift > 0 fails the job and posts a Slack
-ping. Evidence: the workflow file (`.github/workflows/mcp-no-drift.yml`)
-**and** at least one successful run within the last 7 days.
-
-## Promotion procedure
-
-1. Open a release-candidate branch named `release/mcp-beta-rcN`.
-2. Run `./agent-config doctor --check mcp-beta-readiness` — must
-   print all six gates green.
-3. Flip the wording in the **five** surfaces inventoried in the
-   `road-to-surface-discipline` roadmap (Phase 3 Step 1, under
-   `agents/roadmaps/`):
-   `docs/mcp-server.md` (status banner + Remote-MCP sub-claim),
-   `README.md` (pointer line), `scripts/mcp_server/server.py`
-   (initialize-result `serverInfo.name`),
-   `scripts/mcp_server/__init__.py` (module docstring `Stability:`).
-4. Update the changelog with the gate sheet snapshot.
-5. Merge the RC branch through the normal review path. Tag is **not**
-   created until the gate sheet is reproducible on the merge commit.
-
-## Demotion procedure
-
-Any single gate going red on `main` for more than 7 consecutive days
-demotes the surface back to `experimental` at the next release. This
-is a wording-only demotion; no code is reverted. The doctor check
-reports the demotion automatically.
-
-## Surface delta
-
-This contract adds **0 new commands**, **0 new skills**, **0 new
-personas**. It defines a promotion gate; nothing more. Net surface
-delta for Phase 3: ≤ 0.
-
-## Cross-references
-
-- [`mcp-phase-1-scope.md`](mcp-phase-1-scope.md) — local stdio kernel
-  hard contract (A0).
-- [`mcp-cloud-scope.md`](mcp-cloud-scope.md) — hosted Worker hard
-  contract (A0-cloud).
-- [`mcp-tool-stub-envelope.md`](mcp-tool-stub-envelope.md) — Phase 1
-  discovery contract.
-- [`STABILITY.md`](STABILITY.md) — stability tier definitions
-  (`experimental` / `beta` / `stable`) and what wording each tier may
-  use in user-visible surfaces.
-- The `road-to-surface-discipline` roadmap (under `agents/roadmaps/`)
-  — Phase 3 acceptance criteria and step-level evidence pointers.