pattern-mcp 0.3.0 → 0.5.0

package/README.md

@@ -32,7 +32,7 @@ whether to:
 
 Pattern is designed for agents to use **while they are building**.
 
-It exposes four tools:
+It exposes seven tools:
 
 - `recommend_component` — evaluates a UI component need and returns a
   structured recommendation.
@@ -44,7 +44,19 @@ It exposes four tools:
   account.
 - `read_ledger` — lists past `recommend_component` judgments for a
   `project_id`, including any that were served from the ledger cache (see
-  [Per-project judgment ledger](#per-project-judgment-ledger)).
+  [Per-project judgment ledger](#per-project-judgment-ledger)); pass
+  `feature_id` instead of browsing by keyword to get a full cost rollup for
+  one feature (see [Tool: `report_build_cost`](#tool-report_build_cost)).
+- `report_build_cost` — self-reports the end-to-end build cost for one
+  feature, so cost incurred after Pattern's own verdict (the actual
+  scaffold/install/build) is still attributable back to it.
+- `report_outcome_proxy` — self-reports a value signal (rework, time to
+  merge, kept-vs-replaced) for one feature, deliberately independent of
+  Pattern's own verdict -- see [Outcome
+  proxies](#outcome-proxies).
+- `check_ledger_liveness` — checks whether a ledger entry's recorded
+  `file_path` still exists and still references its `chosen_candidate` --
+  see [Tool: `check_ledger_liveness`](#tool-check_ledger_liveness).
 
 ## How it works
 
@@ -409,6 +421,27 @@ Leave `checklist` out to keep today's default behavior: `recommend_component`
 extracts its own checklist internally, exactly as before this option
 existed.
 
+#### `feature_id`
+
+`feature_id` is optional -- a stable identifier for the feature this
+component need belongs to (e.g. a ticket id or branch name). Its only use
+is joining this call's cost with a later
+[`report_build_cost`](#tool-report_build_cost) call for the same feature.
+Omit it to have one derived deterministically from `project_id` +
+`component_need`; only meaningful together with `project_id`. See
+[Feature cost attribution](#feature-cost-attribution).
+
+#### `file_path`
+
+`file_path` is optional -- path (relative to `PROJECT_ROOT`) where this
+component decision is expected to be implemented, if already known.
+Usually not known yet at call time, since the decision typically precedes
+the file existing. When provided, it's stored on the resulting ledger
+entry and [`check_ledger_liveness`](#tool-check_ledger_liveness) can later
+confirm the file still exists and still references `chosen_candidate`. It
+cannot currently be attached to an entry after the fact -- see [Ledger
+integrity and decision provenance](#ledger-integrity-and-decision-provenance).
+
 **Is the checklist actually skipped, not just re-derived?** Checked, not
 assumed. `breakdown_ms.extract` for a `checklist`-provided call is smaller
 than the default path's, but not near-zero -- which raised the question of
@@ -667,6 +700,10 @@ came back with `served_from_ledger: true`.
   match, no embeddings) against stored entries' `component_need`. Omit to
   list everything for the project.
 - `limit` is optional, defaults to 20. Most recent entries first.
+- `feature_id` is optional. When provided, `component_need` and `limit`
+  are ignored and the response is a full cost rollup for that one feature
+  instead of a keyword listing -- see [Feature cost
+  attribution](#feature-cost-attribution).
 
 ### Output
 
@@ -678,6 +715,7 @@ came back with `served_from_ledger: true`.
       "id": "a1b2c3d4-...",
       "timestamp": "2026-08-29T19:50:47.073Z",
       "project_id": "my-booking-app",
+      "feature_id": "3f9a21c0",
       "component_need": "cancellation policy display with refund tiers by date",
       "domain": "Airbnb-style rental marketplace",
       "framework": "React + Tailwind",
@@ -691,24 +729,291 @@ came back with `served_from_ledger: true`.
       "confidence": "low",
       "reason": "scored",
       "coverage": "5/8 (62.5%)",
-      "project_conventions_snapshot": "9f3a1c7e2b0d4f5a"
+      "cost_usd": 0.087,
+      "cache_hit": false,
+      "project_conventions_snapshot": "9f3a1c7e2b0d4f5a",
+      "file_path": null,
+      "snapshot_ref": "a1b2c3d4e5f6...",
+      "last_verified_live": null,
+      "live_status": "unknown"
     }
   ]
 }
 ```
 
+`file_path`/`snapshot_ref`/`last_verified_live`/`live_status` are the
+ledger integrity + decision provenance fields -- see [Ledger integrity and
+decision provenance](#ledger-integrity-and-decision-provenance) and [Tool:
+`check_ledger_liveness`](#tool-check_ledger_liveness). Entries written
+before this feature shipped read back with `file_path`/`snapshot_ref`/
+`last_verified_live` as `null` and `live_status` as `"unknown"` rather
+than missing keys.
+
+Passing `feature_id` instead returns:
+
+```json
+{
+  "project_id": "my-booking-app",
+  "feature_id": "3f9a21c0",
+  "verdict_entries": [ "...same shape as above, filtered to this feature_id..." ],
+  "build_records": [
+    { "id": "...", "timestamp": "...", "project_id": "my-booking-app", "feature_id": "3f9a21c0", "tokens_used": 9000, "cost_usd": 1.25, "outcome": "shipped" }
+  ],
+  "total_cost_usd": 1.34,
+  "outcome_proxy": { "time_to_merge_hours": 3.5, "reworked": true, "days_to_rework": 12, "status_at_30d": "kept" },
+  "outcome_proxy_history": [ "...every raw report_outcome_proxy record for this feature_id, oldest first..." ]
+}
+```
+
+`outcome_proxy` is `null` (and `outcome_proxy_history` an empty array)
+when no `report_outcome_proxy` calls have been made for this feature yet
+-- see [Outcome proxies](#outcome-proxies).
+
 Each entry holds only distilled fields -- `candidates_evaluated` never
 contains raw HTML, full prop tables, or the per-requirement evidence text
 `recommend_component` itself returns. See
 [Data minimization](#data-minimization) below.
 
+## Tool: `report_build_cost`
+
+Self-reports the end-to-end build cost for one feature. Pattern only ever
+sees the cost of judging *what* to use (`recommend_component`'s own
+`_meta.estimated_cost_usd`); everything past that -- the actual scaffold,
+install, or custom build -- happens outside Pattern entirely and Pattern
+has no way to observe it. Call this once, after the calling agent's build
+for a feature is actually complete (shipped, abandoned, or replaced), not
+on every verdict.
+
+### Input
+
+```json
+{
+  "feature_id": "3f9a21c0",
+  "project_id": "my-booking-app",
+  "tokens_used": 9000,
+  "cost_usd": 1.25,
+  "outcome": "shipped"
+}
+```
+
+- `feature_id` is required -- either a value you explicitly passed to an
+  earlier `recommend_component` call for this feature, or (if you didn't)
+  the same value `recommend_component` derives on its own:
+  `sha256(project_id + "::" + component_need, lowercased/trimmed)`
+  truncated to 8 hex characters. When in doubt, call `read_ledger` with
+  just `project_id` and copy the `feature_id` off the relevant entry
+  rather than re-deriving it by hand.
+- `project_id` is optional but recommended -- without it, this record
+  still joins to a `recommend_component` entry by `feature_id` alone, but
+  `read_ledger`'s rollup can't scope it to one project.
+- `tokens_used` is optional.
+- `cost_usd` is required -- your own real number, not Pattern's.
+- `outcome` is required: `"shipped"`, `"abandoned"`, or
+  `"replaced_with_existing"`.
+
+### Output
+
+```json
+{
+  "status": "recorded",
+  "record": {
+    "id": "c5706b47-...",
+    "timestamp": "2026-09-02T01:25:29.653Z",
+    "project_id": "my-booking-app",
+    "feature_id": "3f9a21c0",
+    "tokens_used": 9000,
+    "cost_usd": 1.25,
+    "outcome": "shipped"
+  }
+}
+```
+
+This only appends a local record to `~/.pattern/build_ledger.jsonl`
+(override with `PATTERN_BUILD_LEDGER_PATH`) -- it never re-runs any
+judgment and never calls the Anthropic API.
+
+## Tool: `report_outcome_proxy`
+
+Self-reports a value signal for one feature, deliberately independent of
+Pattern's own verdict -- the whole point is a signal that could
+*contradict* the verdict, so nothing on this path ever reads
+`coverage_pct`, `confidence`, or any other Pattern-produced field. Compute
+`reworked`/`days_to_rework` and `time_to_merge_hours` from your own repo's
+real git history (e.g. `git log --follow` against the files this
+feature's build touched) rather than relying on Pattern -- rework rate and
+time-to-merge need real git *history*, a materially bigger surface than
+the one narrow, read-only exception described in [Ledger integrity and
+decision provenance](#ledger-integrity-and-decision-provenance) below.
+Report `status_at_30d` only once a real ~30-day-post-merge horizon has
+actually passed.
+
+Safe to call more than once for the same `feature_id` as more signal
+becomes available over time -- e.g. `time_to_merge_hours` right after
+merge, `reworked` on a later re-check, `status_at_30d` at the 30-day mark.
+`read_ledger`'s `feature_id` rollup merges every report into one
+latest-value-per-field view (a later report only overwrites the specific
+fields it includes, never the others).
+
+### Input
+
+```json
+{
+  "feature_id": "3f9a21c0",
+  "project_id": "my-booking-app",
+  "reworked": true,
+  "days_to_rework": 12
+}
+```
+
+- `feature_id` is required.
+- `project_id` is optional but recommended, same reasoning as
+  `report_build_cost`.
+- `reworked`, `days_to_rework`, `time_to_merge_hours`, `status_at_30d` are
+  all individually optional, but **at least one is required** -- an empty
+  report is rejected rather than silently recording nothing.
+
+### Output
+
+```json
+{
+  "status": "recorded",
+  "record": {
+    "id": "8a2f1e0c-...",
+    "timestamp": "2026-09-16T18:04:12.881Z",
+    "project_id": "my-booking-app",
+    "feature_id": "3f9a21c0",
+    "reworked": true,
+    "days_to_rework": 12
+  }
+}
+```
+
+This only appends a local record to `~/.pattern/outcome_proxies.jsonl`
+(override with `PATTERN_OUTCOME_PROXY_PATH`) -- it never calls the
+Anthropic API.
+
+## Tool: `check_ledger_liveness`
+
+Checks whether ledger entries for a `project_id` are still **live** --
+does the `file_path` recorded on the entry (if any, see
+[`file_path`](#tool-recommend_component)) still exist, and does it still
+mention `chosen_candidate`. See [Ledger integrity and decision
+provenance](#ledger-integrity-and-decision-provenance) for the full design
+and its deliberate limits.
+
+This is the **one exception** to Pattern otherwise having no filesystem
+access to your repo (see [Outcome proxies](#outcome-proxies) above) --
+scoped narrowly to read-only `fs.existsSync`/file-read calls against
+`PROJECT_ROOT` (defaults to this server's own working directory; override
+with `PATTERN_PROJECT_ROOT`). It never writes to your repo and never runs
+an arbitrary shell command.
+
+### Input
+
+```json
+{
+  "project_id": "my-booking-app",
+  "ledger_entry_id": "a1b2c3d4-..."
+}
+```
+
+- `project_id` is required.
+- `ledger_entry_id` is optional -- check just that one entry instead of
+  every entry for `project_id` that has a `file_path` set.
+
+### Output
+
+```json
+{
+  "project_id": "my-booking-app",
+  "checked": 1,
+  "total_entries": 2,
+  "results": [
+    {
+      "ledger_entry_id": "a1b2c3d4-...",
+      "component_need": "cancellation policy display with refund tiers by date",
+      "file_path": "src/components/CancellationPolicy.tsx",
+      "live_status": "live",
+      "checked_at": "2026-09-02T20:11:03.442Z",
+      "note": null
+    },
+    {
+      "ledger_entry_id": "e5f6a7b8-...",
+      "component_need": "gallery",
+      "file_path": null,
+      "live_status": "unknown",
+      "checked_at": null,
+      "note": "no file_path recorded on this entry -- nothing to check"
+    }
+  ]
+}
+```
+
+`live_status` is one of `"live"`, `"orphaned"`, `"unknown"`, or
+(reserved, not yet produced -- see [Ledger integrity and decision
+provenance](#ledger-integrity-and-decision-provenance)) `"dangling"`.
+Entries with no `file_path` are listed but never checked or written to
+`ledger_liveness.jsonl` -- their status is permanently `"unknown"` since
+there's nothing to check. Results here are also layered onto
+`read_ledger`'s `live_status`/`last_verified_live` fields for the same
+entries afterward -- `check_ledger_liveness` is the only thing that
+advances those fields past their write-time defaults.
+
+## Feature cost attribution
+
+Every `recommend_component` call that writes to the ledger -- a fresh
+judgment *or* a $0 [ledger cache hit](#the-cache-hit-exception) -- now
+carries a `feature_id`, plus its own `cost_usd` and `cache_hit`. Pair that
+with `report_build_cost`'s build-time record and `read_ledger`'s
+`feature_id` rollup, and total spend on a feature (judgment + build,
+across however many calls) is queryable end to end, not just the cost of
+one verdict call.
+
+`feature_id` defaults to a deterministic derivation --
+`sha256(project_id + "::" + component_need)` truncated to 8 hex chars --
+so repeat calls for the same feature land under the same id automatically,
+with no coordination needed between `recommend_component` and
+`report_build_cost` calls. Pass your own `feature_id` explicitly (e.g. a
+ticket id or branch name) if you'd rather key on something stable on your
+own side.
+
+## Outcome proxies
+
+Cost data alone (`feature cost attribution` above) can't answer whether a
+cheaper build was actually *worth it* -- comparing it against Pattern's
+own verdict/`coverage_pct` would be circular, since that's the very thing
+being evaluated. `report_outcome_proxy` attaches a cheap, non-circular
+value signal per `feature_id` instead:
+
+- **`reworked` / `days_to_rework`** (primary proxy) -- was any file this
+  feature's build touched modified again after the original merge, and if
+  so, how soon? Computed from real git history, not Pattern's own data.
+- **`time_to_merge_hours`** (secondary proxy) -- how long the feature
+  took from first commit to merge.
+- **`status_at_30d`** (tertiary, longer-horizon proxy) -- at a ~30-day
+  horizon, does the component Pattern recommended still exist in the
+  codebase, unchanged in kind (`"kept"`), was it swapped for a different
+  approach (`"replaced"`), or removed entirely (`"removed"`)?
+
+`read_ledger`'s `feature_id` rollup returns both `outcome_proxy` (the
+merged latest-value-per-field view) and `outcome_proxy_history` (every
+raw report, in case the timeline itself matters) alongside the cost
+figures from [Feature cost attribution](#feature-cost-attribution) above
+-- so "what did this feature cost end to end, and did it hold up?" is
+answerable from one `read_ledger` call.
+
 ## Per-project judgment ledger
 
 Distinct from [per-project decision memory](#per-project-decision-memory)
 below -- that file only gains an entry when `record_component_decision` is
 explicitly called. The ledger instead gains one entry automatically for
-**every** `recommend_component` call that reaches the API with a
-`project_id` and lands on reason `"scored"` or `"no_candidates_found"`.
+**every** `recommend_component` call with a `project_id` that lands on
+reason `"scored"` or `"no_candidates_found"` -- whether that's a fresh
+call that reached the API, or a $0 [ledger cache
+hit](#the-cache-hit-exception) served without one (`cache_hit: true`,
+`cost_usd: 0`), so a feature's total cost still rolls up correctly even
+once most of its later calls are free. See [Feature cost
+attribution](#feature-cost-attribution).
 
 Pattern stores it locally in:
 
@@ -769,6 +1074,96 @@ convention: a raw or extended object throws rather than silently
 persisting. Run `node scripts/verify-ledger-boundary.mjs` (after
 `npm run build`) to check this boundary directly.
 
+## Ledger integrity and decision provenance
+
+Two gaps in the ledger, surfaced from user feedback: it tracks that a
+decision was made, but not whether the thing it decided about is still
+live in your codebase, and it stores the checklist/verdict but not a
+version pin or an exportable artifact you can attach to a PR or issue.
+This section covers what's shipped so far -- **P0/P1 of both halves**, not
+the full spec. See `pattern-ledger-integrity-and-provenance-spec.md` for
+the complete phased plan; P2/P3 (a scheduled/batch sweep, dangling-cluster
+detection, the provenance-artifact exporter, and GitHub PR/issue posting)
+are not built yet.
+
+**This is the one deliberate exception** to Pattern otherwise having [no
+filesystem/git access to your repo](#per-project-judgment-ledger) at all
+(the principle `report_build_cost`/`report_outcome_proxy` are built
+around). It's narrow on purpose:
+
+- `git rev-parse HEAD` (read-only, never touches repo state) to capture
+  `snapshot_ref` on every ledger write.
+- `fs.existsSync` plus a plain-text read of one file, only for a
+  `file_path` you explicitly passed to `recommend_component`, only inside
+  `PROJECT_ROOT` (see below), to answer `check_ledger_liveness`.
+
+Nothing here runs an arbitrary git or shell command, and nothing writes to
+your repo.
+
+### `PROJECT_ROOT`
+
+Defaults to `process.cwd()` -- for a locally-run stdio MCP server, that's
+normally the consuming repo's root, since MCP hosts typically launch the
+server with the project directory as its working directory. Override with
+`PATTERN_PROJECT_ROOT` if that assumption doesn't hold for your setup.
+
+A `file_path` that's absolute or escapes `PROJECT_ROOT` via `../` resolves
+to `live_status: "unknown"` rather than being read -- belt-and-suspenders,
+since the calling agent already has real filesystem access to its own
+machine regardless.
+
+### Decision provenance: `snapshot_ref`
+
+Every ledger entry -- fresh judgment or [ledger cache
+hit](#the-cache-hit-exception) -- now carries `snapshot_ref`: the commit
+SHA of `PROJECT_ROOT` at the moment that line was written, or `null` when
+`PROJECT_ROOT` isn't a git repo (or `git` isn't installed, or the call
+times out) -- this never fails the underlying `recommend_component` call.
+Entries written before this shipped read back with `snapshot_ref: null`.
+
+A cache-hit entry's `snapshot_ref` reflects the codebase state *when that
+cache-hit line was written*, not the original judgment's -- to see the
+original judgment's snapshot, look up the entry named in its
+`ledger_entry_id`/`original_verdict_timestamp` fields instead.
+
+Not yet built (P1-P3 of Feature 2): a markdown export of one entry's full
+record for attaching to a PR/issue, and the GitHub posting action itself.
+
+### Referential integrity: `file_path` / `live_status`
+
+`recommend_component` optionally accepts `file_path` (see [Tool:
+`recommend_component`](#tool-recommend_component)) -- usually not known at
+call time, since the decision typically precedes the file existing. When
+set, [`check_ledger_liveness`](#tool-check_ledger_liveness) can later
+check whether that file still exists and still mentions
+`chosen_candidate`:
+
+- **`live`** -- the file exists and mentions `chosen_candidate`.
+- **`orphaned`** -- `file_path` is set but the file no longer exists.
+- **`unknown`** -- no `file_path` was ever recorded, the path escapes
+  `PROJECT_ROOT`, or the file exists but `chosen_candidate` can't be
+  confirmed in it. Deliberately the default outcome for anything
+  ambiguous: a false `"orphaned"` is worse than a lingering `"unknown"`.
+- **`dangling`** -- reserved, not yet produced. Feature 1's second
+  staleness type (a cluster of entries that only reference each other,
+  with no live anchor anywhere) is graph-level analysis across the whole
+  ledger, not a single-entry check -- P3, not built here.
+
+Checks are on-demand only right now (call `check_ledger_liveness`
+yourself, or on whatever schedule you want) -- there's no automatic
+sweep. `live_status`/`last_verified_live` start `"unknown"`/`null` on
+every entry at write time and only ever advance via a
+`check_ledger_liveness` call; results are stored append-only in
+`~/.pattern/ledger_liveness.jsonl` (override with
+`PATTERN_LEDGER_LIVENESS_PATH`, same "append, never mutate the source
+line, most recent record wins at read time" convention as
+`outcome_proxies.jsonl`, see [Outcome proxies](#outcome-proxies)) and
+layered onto `ledger.jsonl`'s own entries at read time -- the ledger line
+itself is never rewritten.
+
+Not yet built (P2-P3 of Feature 1): a scheduled/batch sweep across an
+entire ledger, and dangling-cluster detection.
+
 ## Per-project decision memory
 
 Pattern stores confirmed decisions locally in:
@@ -837,9 +1232,75 @@ recommendations.
 Local project memory and the local call log are stored on the machine
 running Pattern. They are not sent anywhere by Pattern itself.
 
+The one exception is opt-in telemetry, off by default -- see
+[Telemetry](#telemetry) below for exactly what it sends and how to turn
+it on or off.
+
 Review [SECURITY.md](./SECURITY.md) before putting sensitive information
 into fields such as `component_need`, `domain`, or project IDs.
 
+## Telemetry
+
+Off by default. Nothing is sent anywhere for telemetry purposes unless
+you explicitly set:
+
+```
+PATTERN_TELEMETRY=1
+```
+
+**The one-time notice.** The first time you run this version of Pattern
+-- whether it's a brand-new install or an upgrade from a version before
+telemetry existed -- it prints a short notice to stderr explaining all of
+this and how to opt in. It prints exactly once, ever (tracked by a marker
+file at `~/.pattern/telemetry_notice_shown`), then never again, regardless
+of whether you act on it. There's no interactive y/n prompt: Pattern's
+stdin is the MCP JSON-RPC channel the client uses to talk to it, so
+blocking on stdin for a keypress would fight the protocol handshake
+instead of showing a dialog -- a stderr notice is the safe equivalent for
+a stdio MCP server.
+
+**Why it exists.** Two things about real usage can't be answered from
+this repo alone: whether people actually come back and use Pattern on a
+second or third project on their own, and how often a BYO Anthropic key
+actually hits a rate limit or runs out of credit in real sessions, not
+just the one time that happened during manual testing (see
+[Known limitations](#known-limitations)). Telemetry answers both without
+requiring anyone to fill out a survey.
+
+**What gets sent, when enabled:**
+
+- An anonymous, randomly generated install ID -- a UUID created once and
+  stored at `~/.pattern/install_id` (overridable via
+  `PATTERN_INSTALL_ID_PATH`), never derived from your machine, username,
+  or any other identifying information. This is the only thing that ties
+  two events together as "the same install."
+- A one-way SHA-256 hash of `project_id`, truncated to 16 hex characters
+  -- never the raw `project_id` string. The hash lets Pattern count how
+  many *distinct* projects one install has used, without ever seeing what
+  those projects are named.
+- On every `recommend_component` call that reaches the API or the ledger
+  cache-hit shortcut: `verdict`, `confidence`, `reason`,
+  `ensemble_triggered`, `estimated_cost_usd`, and `served_from_ledger` --
+  the same distilled shape already written to the
+  [local call log](#local-call-log), not new information.
+- On a failed Anthropic API call specifically: the HTTP status code and a
+  coarse classification (`rate_limit`, `insufficient_credit`, or `other`)
+  -- never the request or response body.
+
+**What never gets sent, telemetry on or off:** `component_need`,
+`domain`, `framework`, `existing_stack`, `requirements_checked` evidence,
+the raw `project_id`, or your Anthropic API key.
+
+**Where it goes.** Events go to Pattern's PostHog project via its public,
+write-only project key (safe to ship in source -- it can send events, it
+cannot read data back). Set `PATTERN_POSTHOG_KEY` /
+`PATTERN_POSTHOG_HOST` to point at a different project, e.g. for
+self-hosting.
+
+**Turning it off** is the default -- just don't set `PATTERN_TELEMETRY`.
+If you'd previously enabled it, unset the variable (or set it to `0`) to
+go back to fully local.
+
 ## Cost
 
 Pattern uses the Anthropic API, so `recommend_component` has a cost.