pattern-mcp 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -32,7 +32,7 @@ whether to:
32
32
 
33
33
  Pattern is designed for agents to use **while they are building**.
34
34
 
35
- It exposes four tools:
35
+ It exposes six tools:
36
36
 
37
37
  - `recommend_component` — evaluates a UI component need and returns a
38
38
  structured recommendation.
@@ -44,7 +44,16 @@ It exposes four tools:
44
44
  account.
45
45
  - `read_ledger` — lists past `recommend_component` judgments for a
46
46
  `project_id`, including any that were served from the ledger cache (see
47
- [Per-project judgment ledger](#per-project-judgment-ledger)).
47
+ [Per-project judgment ledger](#per-project-judgment-ledger)); pass
48
+ `feature_id` instead of browsing by keyword to get a full cost rollup for
49
+ one feature (see [Tool: `report_build_cost`](#tool-report_build_cost)).
50
+ - `report_build_cost` — self-reports the end-to-end build cost for one
51
+ feature, so cost incurred after Pattern's own verdict (the actual
52
+ scaffold/install/build) is still attributable back to it.
53
+ - `report_outcome_proxy` — self-reports a value signal (rework, time to
54
+ merge, kept-vs-replaced) for one feature, deliberately independent of
55
+ Pattern's own verdict -- see [Outcome
56
+ proxies](#outcome-proxies).
48
57
 
49
58
  ## How it works
50
59
 
@@ -409,6 +418,16 @@ Leave `checklist` out to keep today's default behavior: `recommend_component`
409
418
  extracts its own checklist internally, exactly as before this option
410
419
  existed.
411
420
 
421
+ #### `feature_id`
422
+
423
+ `feature_id` is optional -- a stable identifier for the feature this
424
+ component need belongs to (e.g. a ticket id or branch name). Its only use
425
+ is joining this call's cost with a later
426
+ [`report_build_cost`](#tool-report_build_cost) call for the same feature.
427
+ Omit it to have one derived deterministically from `project_id` +
428
+ `component_need`; only meaningful together with `project_id`. See
429
+ [Feature cost attribution](#feature-cost-attribution).
430
+
412
431
  **Is the checklist actually skipped, not just re-derived?** Checked, not
413
432
  assumed. `breakdown_ms.extract` for a `checklist`-provided call is smaller
414
433
  than the default path's, but not near-zero -- which raised the question of
@@ -667,6 +686,10 @@ came back with `served_from_ledger: true`.
667
686
  match, no embeddings) against stored entries' `component_need`. Omit to
668
687
  list everything for the project.
669
688
  - `limit` is optional, defaults to 20. Most recent entries first.
689
+ - `feature_id` is optional. When provided, `component_need` and `limit`
690
+ are ignored and the response is a full cost rollup for that one feature
691
+ instead of a keyword listing -- see [Feature cost
692
+ attribution](#feature-cost-attribution).
670
693
 
671
694
  ### Output
672
695
 
@@ -678,6 +701,7 @@ came back with `served_from_ledger: true`.
678
701
  "id": "a1b2c3d4-...",
679
702
  "timestamp": "2026-08-29T19:50:47.073Z",
680
703
  "project_id": "my-booking-app",
704
+ "feature_id": "3f9a21c0",
681
705
  "component_need": "cancellation policy display with refund tiers by date",
682
706
  "domain": "Airbnb-style rental marketplace",
683
707
  "framework": "React + Tailwind",
@@ -691,24 +715,210 @@ came back with `served_from_ledger: true`.
691
715
  "confidence": "low",
692
716
  "reason": "scored",
693
717
  "coverage": "5/8 (62.5%)",
718
+ "cost_usd": 0.087,
719
+ "cache_hit": false,
694
720
  "project_conventions_snapshot": "9f3a1c7e2b0d4f5a"
695
721
  }
696
722
  ]
697
723
  }
698
724
  ```
699
725
 
726
+ Passing `feature_id` instead returns:
727
+
728
+ ```json
729
+ {
730
+ "project_id": "my-booking-app",
731
+ "feature_id": "3f9a21c0",
732
+ "verdict_entries": [ "...same shape as above, filtered to this feature_id..." ],
733
+ "build_records": [
734
+ { "id": "...", "timestamp": "...", "project_id": "my-booking-app", "feature_id": "3f9a21c0", "tokens_used": 9000, "cost_usd": 1.25, "outcome": "shipped" }
735
+ ],
736
+ "total_cost_usd": 1.34,
737
+ "outcome_proxy": { "time_to_merge_hours": 3.5, "reworked": true, "days_to_rework": 12, "status_at_30d": "kept" },
738
+ "outcome_proxy_history": [ "...every raw report_outcome_proxy record for this feature_id, oldest first..." ]
739
+ }
740
+ ```
741
+
742
+ `outcome_proxy` is `null` (and `outcome_proxy_history` an empty array)
743
+ when no `report_outcome_proxy` calls have been made for this feature yet
744
+ -- see [Outcome proxies](#outcome-proxies).
745
+
700
746
  Each entry holds only distilled fields -- `candidates_evaluated` never
701
747
  contains raw HTML, full prop tables, or the per-requirement evidence text
702
748
  `recommend_component` itself returns. See
703
749
  [Data minimization](#data-minimization) below.
704
750
 
751
+ ## Tool: `report_build_cost`
752
+
753
+ Self-reports the end-to-end build cost for one feature. Pattern only ever
754
+ sees the cost of judging *what* to use (`recommend_component`'s own
755
+ `_meta.estimated_cost_usd`); everything past that -- the actual scaffold,
756
+ install, or custom build -- happens outside Pattern entirely and Pattern
757
+ has no way to observe it. Call this once, after the calling agent's build
758
+ for a feature is actually complete (shipped, abandoned, or replaced), not
759
+ on every verdict.
760
+
761
+ ### Input
762
+
763
+ ```json
764
+ {
765
+ "feature_id": "3f9a21c0",
766
+ "project_id": "my-booking-app",
767
+ "tokens_used": 9000,
768
+ "cost_usd": 1.25,
769
+ "outcome": "shipped"
770
+ }
771
+ ```
772
+
773
+ - `feature_id` is required -- either a value you explicitly passed to an
774
+ earlier `recommend_component` call for this feature, or (if you didn't)
775
+ the same value `recommend_component` derives on its own:
776
+ `sha256(project_id + "::" + component_need, lowercased/trimmed)`
777
+ truncated to 8 hex characters. When in doubt, call `read_ledger` with
778
+ just `project_id` and copy the `feature_id` off the relevant entry
779
+ rather than re-deriving it by hand.
780
+ - `project_id` is optional but recommended -- without it, this record
781
+ still joins to a `recommend_component` entry by `feature_id` alone, but
782
+ `read_ledger`'s rollup can't scope it to one project.
783
+ - `tokens_used` is optional.
784
+ - `cost_usd` is required -- your own real number, not Pattern's.
785
+ - `outcome` is required: `"shipped"`, `"abandoned"`, or
786
+ `"replaced_with_existing"`.
787
+
788
+ ### Output
789
+
790
+ ```json
791
+ {
792
+ "status": "recorded",
793
+ "record": {
794
+ "id": "c5706b47-...",
795
+ "timestamp": "2026-09-02T01:25:29.653Z",
796
+ "project_id": "my-booking-app",
797
+ "feature_id": "3f9a21c0",
798
+ "tokens_used": 9000,
799
+ "cost_usd": 1.25,
800
+ "outcome": "shipped"
801
+ }
802
+ }
803
+ ```
804
+
805
+ This only appends a local record to `~/.pattern/build_ledger.jsonl`
806
+ (override with `PATTERN_BUILD_LEDGER_PATH`) -- it never re-runs any
807
+ judgment and never calls the Anthropic API.
808
+
809
+ ## Tool: `report_outcome_proxy`
810
+
811
+ Self-reports a value signal for one feature, deliberately independent of
812
+ Pattern's own verdict -- the whole point is a signal that could
813
+ *contradict* the verdict, so nothing on this path ever reads
814
+ `coverage_pct`, `confidence`, or any other Pattern-produced field. Compute
815
+ `reworked`/`days_to_rework` and `time_to_merge_hours` from your own repo's
816
+ real git history (e.g. `git log --follow` against the files this
817
+ feature's build touched) -- Pattern has no `process.cwd()`/repo-path
818
+ concept and no filesystem access to your repo at all, so it can't compute
819
+ these itself. Report `status_at_30d` only once a real ~30-day-post-merge
820
+ horizon has actually passed.
821
+
822
+ Safe to call more than once for the same `feature_id` as more signal
823
+ becomes available over time -- e.g. `time_to_merge_hours` right after
824
+ merge, `reworked` on a later re-check, `status_at_30d` at the 30-day mark.
825
+ `read_ledger`'s `feature_id` rollup merges every report into one
826
+ latest-value-per-field view (a later report only overwrites the specific
827
+ fields it includes, never the others).
828
+
829
+ ### Input
830
+
831
+ ```json
832
+ {
833
+ "feature_id": "3f9a21c0",
834
+ "project_id": "my-booking-app",
835
+ "reworked": true,
836
+ "days_to_rework": 12
837
+ }
838
+ ```
839
+
840
+ - `feature_id` is required.
841
+ - `project_id` is optional but recommended, same reasoning as
842
+ `report_build_cost`.
843
+ - `reworked`, `days_to_rework`, `time_to_merge_hours`, `status_at_30d` are
844
+ all individually optional, but **at least one is required** -- an empty
845
+ report is rejected rather than silently recording nothing.
846
+
847
+ ### Output
848
+
849
+ ```json
850
+ {
851
+ "status": "recorded",
852
+ "record": {
853
+ "id": "8a2f1e0c-...",
854
+ "timestamp": "2026-09-16T18:04:12.881Z",
855
+ "project_id": "my-booking-app",
856
+ "feature_id": "3f9a21c0",
857
+ "reworked": true,
858
+ "days_to_rework": 12
859
+ }
860
+ }
861
+ ```
862
+
863
+ This only appends a local record to `~/.pattern/outcome_proxies.jsonl`
864
+ (override with `PATTERN_OUTCOME_PROXY_PATH`) -- it never calls the
865
+ Anthropic API.
866
+
867
+ ## Feature cost attribution
868
+
869
+ Every `recommend_component` call that writes to the ledger -- a fresh
870
+ judgment *or* a $0 [ledger cache hit](#the-cache-hit-exception) -- now
871
+ carries a `feature_id`, plus its own `cost_usd` and `cache_hit`. Pair that
872
+ with `report_build_cost`'s build-time record and `read_ledger`'s
873
+ `feature_id` rollup, and total spend on a feature (judgment + build,
874
+ across however many calls) is queryable end to end, not just the cost of
875
+ one verdict call.
876
+
877
+ `feature_id` defaults to a deterministic derivation --
878
+ `sha256(project_id + "::" + component_need)` truncated to 8 hex chars --
879
+ so repeat calls for the same feature land under the same id automatically,
880
+ with no coordination needed between `recommend_component` and
881
+ `report_build_cost` calls. Pass your own `feature_id` explicitly (e.g. a
882
+ ticket id or branch name) if you'd rather key on something stable on your
883
+ own side.
884
+
885
+ ## Outcome proxies
886
+
887
+ Cost data alone (`feature cost attribution` above) can't answer whether a
888
+ cheaper build was actually *worth it* -- comparing it against Pattern's
889
+ own verdict/`coverage_pct` would be circular, since that's the very thing
890
+ being evaluated. `report_outcome_proxy` attaches a cheap, non-circular
891
+ value signal per `feature_id` instead:
892
+
893
+ - **`reworked` / `days_to_rework`** (primary proxy) -- was any file this
894
+ feature's build touched modified again after the original merge, and if
895
+ so, how soon? Computed from real git history, not Pattern's own data.
896
+ - **`time_to_merge_hours`** (secondary proxy) -- how long the feature
897
+ took from first commit to merge.
898
+ - **`status_at_30d`** (tertiary, longer-horizon proxy) -- at a ~30-day
899
+ horizon, does the component Pattern recommended still exist in the
900
+ codebase, unchanged in kind (`"kept"`), was it swapped for a different
901
+ approach (`"replaced"`), or removed entirely (`"removed"`)?
902
+
903
+ `read_ledger`'s `feature_id` rollup returns both `outcome_proxy` (the
904
+ merged latest-value-per-field view) and `outcome_proxy_history` (every
905
+ raw report, in case the timeline itself matters) alongside the cost
906
+ figures from [Feature cost attribution](#feature-cost-attribution) above
907
+ -- so "what did this feature cost end to end, and did it hold up?" is
908
+ answerable from one `read_ledger` call.
909
+
705
910
  ## Per-project judgment ledger
706
911
 
707
912
  Distinct from [per-project decision memory](#per-project-decision-memory)
708
913
  below -- that file only gains an entry when `record_component_decision` is
709
914
  explicitly called. The ledger instead gains one entry automatically for
710
- **every** `recommend_component` call that reaches the API with a
711
- `project_id` and lands on reason `"scored"` or `"no_candidates_found"`.
915
+ **every** `recommend_component` call with a `project_id` that lands on
916
+ reason `"scored"` or `"no_candidates_found"` -- whether that's a fresh
917
+ call that reached the API, or a $0 [ledger cache
918
+ hit](#the-cache-hit-exception) served without one (`cache_hit: true`,
919
+ `cost_usd: 0`), so a feature's total cost still rolls up correctly even
920
+ once most of its later calls are free. See [Feature cost
921
+ attribution](#feature-cost-attribution).
712
922
 
713
923
  Pattern stores it locally in:
714
924
 
@@ -837,9 +1047,75 @@ recommendations.
837
1047
  Local project memory and the local call log are stored on the machine
838
1048
  running Pattern. They are not sent anywhere by Pattern itself.
839
1049
 
1050
+ The one exception is opt-in telemetry, off by default -- see
1051
+ [Telemetry](#telemetry) below for exactly what it sends and how to turn
1052
+ it on or off.
1053
+
840
1054
  Review [SECURITY.md](./SECURITY.md) before putting sensitive information
841
1055
  into fields such as `component_need`, `domain`, or project IDs.
842
1056
 
1057
+ ## Telemetry
1058
+
1059
+ Off by default. Nothing is sent anywhere for telemetry purposes unless
1060
+ you explicitly set:
1061
+
1062
+ ```
1063
+ PATTERN_TELEMETRY=1
1064
+ ```
1065
+
1066
+ **The one-time notice.** The first time you run this version of Pattern
1067
+ -- whether it's a brand-new install or an upgrade from a version before
1068
+ telemetry existed -- it prints a short notice to stderr explaining all of
1069
+ this and how to opt in. It prints exactly once, ever (tracked by a marker
1070
+ file at `~/.pattern/telemetry_notice_shown`), then never again, regardless
1071
+ of whether you act on it. There's no interactive y/n prompt: Pattern's
1072
+ stdin is the MCP JSON-RPC channel the client uses to talk to it, so
1073
+ blocking on stdin for a keypress would fight the protocol handshake
1074
+ instead of showing a dialog -- a stderr notice is the safe equivalent for
1075
+ a stdio MCP server.
1076
+
1077
+ **Why it exists.** Two things about real usage can't be answered from
1078
+ this repo alone: whether people actually come back and use Pattern on a
1079
+ second or third project on their own, and how often a BYO Anthropic key
1080
+ actually hits a rate limit or runs out of credit in real sessions, not
1081
+ just the one time that happened during manual testing (see
1082
+ [Known limitations](#known-limitations)). Telemetry answers both without
1083
+ requiring anyone to fill out a survey.
1084
+
1085
+ **What gets sent, when enabled:**
1086
+
1087
+ - An anonymous, randomly generated install ID -- a UUID created once and
1088
+ stored at `~/.pattern/install_id` (overridable via
1089
+ `PATTERN_INSTALL_ID_PATH`), never derived from your machine, username,
1090
+ or any other identifying information. This is the only thing that ties
1091
+ two events together as "the same install."
1092
+ - A one-way SHA-256 hash of `project_id`, truncated to 16 hex characters
1093
+ -- never the raw `project_id` string. The hash lets Pattern count how
1094
+ many *distinct* projects one install has used, without ever seeing what
1095
+ those projects are named.
1096
+ - On every `recommend_component` call that reaches the API or the ledger
1097
+ cache-hit shortcut: `verdict`, `confidence`, `reason`,
1098
+ `ensemble_triggered`, `estimated_cost_usd`, and `served_from_ledger` --
1099
+ the same distilled shape already written to the
1100
+ [local call log](#local-call-log), not new information.
1101
+ - On a failed Anthropic API call specifically: the HTTP status code and a
1102
+ coarse classification (`rate_limit`, `insufficient_credit`, or `other`)
1103
+ -- never the request or response body.
1104
+
1105
+ **What never gets sent, telemetry on or off:** `component_need`,
1106
+ `domain`, `framework`, `existing_stack`, `requirements_checked` evidence,
1107
+ the raw `project_id`, or your Anthropic API key.
1108
+
1109
+ **Where it goes.** Events go to Pattern's PostHog project via its public,
1110
+ write-only project key (safe to ship in source -- it can send events, it
1111
+ cannot read data back). Set `PATTERN_POSTHOG_KEY` /
1112
+ `PATTERN_POSTHOG_HOST` to point at a different project, e.g. for
1113
+ self-hosting.
1114
+
1115
+ **Turning it off** is the default -- just don't set `PATTERN_TELEMETRY`.
1116
+ If you'd previously enabled it, unset the variable (or set it to `0`) to
1117
+ go back to fully local.
1118
+
843
1119
  ## Cost
844
1120
 
845
1121
  Pattern uses the Anthropic API, so `recommend_component` has a cost.
package/dist/index.js CHANGED
@@ -34,6 +34,7 @@ import { createHash, randomUUID } from "node:crypto";
34
34
  import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
35
35
  import { homedir } from "node:os";
36
36
  import { dirname, join } from "node:path";
37
+ import { captureApiError, captureRecommendation, printTelemetryNoticeOnce, shutdownTelemetry, } from "./telemetry.js";
37
38
  export const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY;
38
39
  // Configurable so Sonnet vs. Haiku can be A/B tested without a code change.
39
40
  // Defaults to Sonnet 5. Try MODEL=claude-haiku-4-5-20251001 to test the
@@ -227,12 +228,16 @@ export function computeBreakdownMs(t) {
227
228
  };
228
229
  }
229
230
  function buildMeta(timings, usage) {
231
+ const fresh = usage.input_tokens ?? 0;
232
+ const cacheWrite = usage.cache_creation_input_tokens ?? 0;
233
+ const cacheRead = usage.cache_read_input_tokens ?? 0;
230
234
  return {
231
235
  total_ms: timings.scoreEndMs - timings.requestStartMs,
232
236
  breakdown_ms: computeBreakdownMs(timings),
233
237
  tokens_used: {
234
- input: (usage.input_tokens ?? 0) + (usage.cache_creation_input_tokens ?? 0) + (usage.cache_read_input_tokens ?? 0),
238
+ input: fresh + cacheWrite + cacheRead,
235
239
  output: usage.output_tokens ?? 0,
240
+ input_breakdown: { fresh, cache_write: cacheWrite, cache_read: cacheRead },
236
241
  },
237
242
  estimated_cost_usd: estimateCostUsd(usage, MODEL),
238
243
  };
@@ -375,6 +380,8 @@ const TOOL_NAME = "recommend_component";
375
380
  const RECORD_DECISION_TOOL_NAME = "record_component_decision";
376
381
  const EXTRACT_REQUIREMENTS_TOOL_NAME = "extract_requirements";
377
382
  const READ_LEDGER_TOOL_NAME = "read_ledger";
383
+ const REPORT_BUILD_COST_TOOL_NAME = "report_build_cost";
384
+ const REPORT_OUTCOME_PROXY_TOOL_NAME = "report_outcome_proxy";
378
385
  const INPUT_SCHEMA = {
379
386
  type: "object",
380
387
  properties: {
@@ -424,6 +431,16 @@ const INPUT_SCHEMA = {
424
431
  "today's default behavior: recommend_component extracts its own " +
425
432
  "checklist internally, unchanged.",
426
433
  },
434
+ feature_id: {
435
+ type: "string",
436
+ description: "Optional. A stable identifier for the feature this component need " +
437
+ "belongs to (e.g. a ticket id or branch name), used to roll up this " +
438
+ "call's cost with a later report_build_cost call for the same " +
439
+ "feature. Omit to have one derived deterministically from " +
440
+ "project_id+component_need -- repeat calls for the same feature " +
441
+ "then land under the same id automatically, with no coordination " +
442
+ "needed between calls. Only meaningful together with project_id.",
443
+ },
427
444
  },
428
445
  required: ["component_need", "domain", "framework"],
429
446
  };
@@ -497,9 +514,84 @@ const READ_LEDGER_INPUT_SCHEMA = {
497
514
  type: "number",
498
515
  description: "Optional. Maximum number of entries to return, most recent first. Defaults to 20.",
499
516
  },
517
+ feature_id: {
518
+ type: "string",
519
+ description: "Optional. Instead of the usual keyword listing, returns the full " +
520
+ "cost rollup for this one feature_id -- every verdict-time ledger " +
521
+ "entry (fresh judgments and $0 ledger cache hits) plus every " +
522
+ "report_build_cost record for it, with a summed total_cost_usd. " +
523
+ "When provided, component_need and limit are ignored.",
524
+ },
500
525
  },
501
526
  required: ["project_id"],
502
527
  };
528
+ const REPORT_BUILD_COST_INPUT_SCHEMA = {
529
+ type: "object",
530
+ properties: {
531
+ feature_id: {
532
+ type: "string",
533
+ description: "The feature_id this build belongs to -- either one you explicitly " +
534
+ "passed to an earlier recommend_component call for this feature, " +
535
+ "or (if you didn't) the same value recommend_component would " +
536
+ "derive on its own: sha256(project_id + '::' + component_need, " +
537
+ "lowercased/trimmed) truncated to 8 hex chars. When in doubt, call " +
538
+ "read_ledger with just project_id and copy the feature_id off the " +
539
+ "relevant entry rather than re-deriving it by hand.",
540
+ },
541
+ project_id: {
542
+ type: "string",
543
+ description: "Optional but recommended. The same project_id used in the recommend_component call(s) for this feature, so read_ledger's feature_id rollup can find this record.",
544
+ },
545
+ tokens_used: {
546
+ type: "number",
547
+ description: "Optional. Total tokens spent building this feature, if you have a real number (e.g. from your own session accounting).",
548
+ },
549
+ cost_usd: {
550
+ type: "number",
551
+ description: "Total real spend, in USD, for building this feature end to end -- your own best number, not Pattern's (Pattern has no visibility past the verdict it returned).",
552
+ },
553
+ outcome: {
554
+ type: "string",
555
+ enum: ["shipped", "abandoned", "replaced_with_existing"],
556
+ description: "What actually happened to this build: 'shipped' it went out, " +
557
+ "'abandoned' the build was dropped before shipping, " +
558
+ "'replaced_with_existing' you started a custom build but swapped " +
559
+ "in an existing component instead (or vice versa).",
560
+ },
561
+ },
562
+ required: ["feature_id", "cost_usd", "outcome"],
563
+ };
564
+ const REPORT_OUTCOME_PROXY_INPUT_SCHEMA = {
565
+ type: "object",
566
+ properties: {
567
+ feature_id: {
568
+ type: "string",
569
+ description: "The feature_id this outcome data belongs to -- same value used in the feature's recommend_component/report_build_cost calls.",
570
+ },
571
+ project_id: {
572
+ type: "string",
573
+ description: "Optional but recommended. The same project_id used in this feature's other calls, so read_ledger's feature_id rollup can find this record.",
574
+ },
575
+ reworked: {
576
+ type: "boolean",
577
+ description: "Whether any of the files this feature's build touched have been modified again since the original merge -- computed by you from your own repo's git history (e.g. `git log --follow` against the file list), never guessed. Re-report this on a later check if the answer changes.",
578
+ },
579
+ days_to_rework: {
580
+ type: "number",
581
+ description: "Optional. Days between the original merge and the first rework commit, if reworked is true and you have a real date to compute from.",
582
+ },
583
+ time_to_merge_hours: {
584
+ type: "number",
585
+ description: "Hours between the first commit touching this feature's files and the commit/PR that merged it, computed from your own repo's git metadata.",
586
+ },
587
+ status_at_30d: {
588
+ type: "string",
589
+ enum: ["kept", "replaced", "removed"],
590
+ description: "At a ~30-day horizon post-merge: whether the component Pattern recommended still exists in the codebase, unchanged in kind ('kept'), was swapped for a different approach ('replaced'), or was deleted entirely ('removed'). Only report this once the horizon has actually passed.",
591
+ },
592
+ },
593
+ required: ["feature_id"],
594
+ };
503
595
  // Shared between buildSystemPrompt's own step 2 and
504
596
  // buildExtractionSystemPrompt (the extract_requirements tool's standalone
505
597
  // prompt) -- the extraction *instructions* are one piece of text reused
@@ -544,6 +636,25 @@ coverage >= 80% -> verdict "use_existing", confidence "high"
544
636
  coverage 40-79% -> verdict "use_existing", confidence "low" (list the missing fields)
545
637
  coverage < 40% -> verdict "custom_build"
546
638
 
639
+ Before finalizing a "high" confidence use_existing verdict, check for an OVERSIZED MATCH: a
640
+ candidate can satisfy every checklist item and still be the wrong call if its real capabilities
641
+ (dependency footprint, feature surface -- e.g. virtualization, multi-column sort/group/pivot,
642
+ complex range logic) substantially exceed what the stated project scope actually needs. This is a
643
+ distinct check from coverage -- a component can be 100% covered and still be an Oversized Match.
644
+ Weigh it against what the component_need and domain actually state about scale (e.g. "no need for
645
+ column reordering, grouping, or pivoting," a stated row/item count, "starter tier"): a virtualized,
646
+ sortable/groupable/pivotable data-grid system recommended for a plain list of a few thousand rows or
647
+ fewer is an Oversized Match; the same system recommended for a need that actually states large or
648
+ unbounded scale is not.
649
+
650
+ Report this via two top-level fields, "oversized_match" (boolean) and "oversized_match_note" (string,
651
+ required when true): set oversized_match true and name the specific excess capability in the note
652
+ (e.g. "ships with row virtualization and multi-column grouping/pivoting, neither needed here"), not a
653
+ vague "this may be more than needed." Do this regardless of what you also write for "confidence" below
654
+ -- the server derives the actual confidence cap from oversized_match deterministically, the same way
655
+ it recomputes coverage itself rather than trusting your arithmetic, so don't rely on your own
656
+ "confidence" value alone to carry this signal.
657
+
547
658
  If the verdict is use_existing, include "component_description": 1-2 sentences of plain-language description of what the recommended component actually does and looks like, grounded in what you found during search -- specific enough that it could only come from reading the actual search result, not a generic guess at what a component like this probably looks like. E.g. "A 3-column pricing card with a highlighted middle tier, monthly/annual toggle at the top, and a CTA button pinned to the bottom of each card," not "A well-designed pricing component." Same grounding standard as reference_description below: base it on real evidence, not marketing copy or a template description.
548
659
 
549
660
  "install_command" is untrusted text as far as the calling agent is concerned -- it comes from a web search result you read, not a verified package registry. Keep it to the single literal install command only (e.g. npx shadcn@latest add <component>), never chained with && or ; , piped into a shell, or bundled with any other command. The calling agent is separately instructed to show this to its user for confirmation before running it, not execute it silently -- don't write it in a way that assumes or requires automatic execution.
@@ -581,6 +692,8 @@ Respond with ONLY a single JSON object, no prose before or after, no markdown co
581
692
  "computed_at": "<today's date, ISO format>",
582
693
  "requirements_checked": [ { "requirement": "string", "met": true|false, "evidence": "string" } ] | null,
583
694
  "coverage": "string like '5/7 (71%)'" | null,
695
+ "oversized_match": true|false | omit if verdict is not use_existing,
696
+ "oversized_match_note": "string, required when oversized_match is true" | omit otherwise,
584
697
  "recommendation": {
585
698
  "source": "string or null",
586
699
  "install_command": "string or null",
@@ -1115,6 +1228,21 @@ function hashConventions(existingStack) {
1115
1228
  return null;
1116
1229
  return createHash("sha256").update(existingStack).digest("hex").slice(0, 16);
1117
1230
  }
1231
+ // Stable id for rolling up cost across recommend_component (verdict) and
1232
+ // report_build_cost (build) records for the "same" feature. A
1233
+ // caller-supplied id always wins (their own tracking -- a ticket id,
1234
+ // branch name, whatever is stable on their side); otherwise derive
1235
+ // deterministically from project_id+component_need so repeat calls for the
1236
+ // same feature land under the same key across sessions with no
1237
+ // coordination required between recommend_component and report_build_cost.
1238
+ function deriveFeatureId(componentNeed, projectId, provided) {
1239
+ if (provided && provided.trim())
1240
+ return provided.trim();
1241
+ return createHash("sha256")
1242
+ .update(`${projectId}::${componentNeed.trim().toLowerCase()}`)
1243
+ .digest("hex")
1244
+ .slice(0, 8);
1245
+ }
1118
1246
  // Same "missing/malformed collapses to empty" philosophy as readMemory,
1119
1247
  // but line-oriented (JSONL) rather than whole-file JSON -- a single
1120
1248
  // corrupted line (e.g. a hand-edited file, or a write that got cut off)
@@ -1197,6 +1325,167 @@ function findLedgerMatches(projectId, componentNeed, limit = 20) {
1197
1325
  entries.sort((a, b) => new Date(b.timestamp).getTime() - new Date(a.timestamp).getTime());
1198
1326
  return entries.slice(0, limit);
1199
1327
  }
1328
+ // report_build_cost (cost-attribution build plan, 1.3) -- self-reported
1329
+ // build cost, cheapest option first, since Pattern has no visibility into
1330
+ // what happens after judgeComponent returns a verdict (1.4's
1331
+ // session-correlation fallback is a research spike only, not built here).
1332
+ // Stored as a second, separate JSONL file rather than mixed into
1333
+ // ledger.jsonl's LedgerEntry shape -- a BuildRecord has none of
1334
+ // LedgerEntry's verdict/coverage/candidate fields, and keeping the file
1335
+ // single-shape keeps read_ledger's existing output stable. Joined to
1336
+ // verdict records purely by feature_id, per the build plan's data model.
1337
+ const BUILD_LEDGER_PATH = process.env.PATTERN_BUILD_LEDGER_PATH ?? join(homedir(), ".pattern", "build_ledger.jsonl");
1338
+ function appendBuildRecord(record) {
1339
+ mkdirSync(dirname(BUILD_LEDGER_PATH), { recursive: true });
1340
+ appendFileSync(BUILD_LEDGER_PATH, JSON.stringify(record) + "\n", "utf8");
1341
+ }
1342
+ // Same "missing/malformed collapses to empty, one bad line skipped not
1343
+ // fatal" philosophy as readLedgerEntries.
1344
+ function readBuildRecords(featureId) {
1345
+ let raw;
1346
+ try {
1347
+ raw = readFileSync(BUILD_LEDGER_PATH, "utf8");
1348
+ }
1349
+ catch {
1350
+ return [];
1351
+ }
1352
+ const records = [];
1353
+ for (const line of raw.split("\n")) {
1354
+ if (!line.trim())
1355
+ continue;
1356
+ try {
1357
+ const parsed = JSON.parse(line);
1358
+ if (parsed && typeof parsed === "object" && parsed.feature_id === featureId) {
1359
+ records.push(parsed);
1360
+ }
1361
+ }
1362
+ catch {
1363
+ // skip malformed line
1364
+ }
1365
+ }
1366
+ return records;
1367
+ }
1368
+ function recordBuildCost(input) {
1369
+ const record = {
1370
+ id: randomUUID(),
1371
+ timestamp: new Date().toISOString(),
1372
+ project_id: input.project_id,
1373
+ feature_id: input.feature_id,
1374
+ tokens_used: typeof input.tokens_used === "number" && Number.isFinite(input.tokens_used) ? input.tokens_used : null,
1375
+ cost_usd: input.cost_usd,
1376
+ outcome: input.outcome,
1377
+ };
1378
+ appendBuildRecord(record);
1379
+ return record;
1380
+ }
1381
+ // The "total cost per feature is queryable" rollup task 1.5 validates
1382
+ // against a hand total: every verdict-time ledger entry for this
1383
+ // project_id+feature_id (fresh judgments and $0 cache hits alike) plus
1384
+ // every self-reported build record for the same feature_id. project_id is
1385
+ // required, same as every other read here, so this never falls back to a
1386
+ // shared/global bucket across projects.
1387
+ // report_outcome_proxy (cost-attribution build plan Phase 2, 2.1-2.3) --
1388
+ // self-reported, same reasoning as report_build_cost: rework-rate and
1389
+ // time-to-merge both require real git history, and Pattern has no
1390
+ // process.cwd()/repo-path concept and no filesystem access to a caller's
1391
+ // repo at all (see project judgment ledger's own design notes) -- rather
1392
+ // than giving Pattern a new git-shelling-out capability, the calling
1393
+ // agent (which already has real repo access) computes these off its own
1394
+ // `git log`/`git blame` and reports the result here. This also makes
1395
+ // 2.4's exclusion check true by construction: nothing on this path ever
1396
+ // reads coverage_pct, confidence, or any other Pattern-produced field --
1397
+ // there simply isn't a code path from a verdict into an outcome proxy.
1398
+ // Append-only like every other record here: a feature can get multiple
1399
+ // proxy reports over time (time_to_merge_hours right after merge,
1400
+ // reworked/days_to_rework on a later re-check, status_at_30d once the
1401
+ // horizon passes) -- readers take the latest report per field via
1402
+ // latestOutcomeProxy below, not a running mutation of one row.
1403
+ const OUTCOME_PROXY_PATH = process.env.PATTERN_OUTCOME_PROXY_PATH ?? join(homedir(), ".pattern", "outcome_proxies.jsonl");
1404
+ function appendOutcomeProxyRecord(record) {
1405
+ mkdirSync(dirname(OUTCOME_PROXY_PATH), { recursive: true });
1406
+ appendFileSync(OUTCOME_PROXY_PATH, JSON.stringify(record) + "\n", "utf8");
1407
+ }
1408
+ function readOutcomeProxyRecords(featureId) {
1409
+ let raw;
1410
+ try {
1411
+ raw = readFileSync(OUTCOME_PROXY_PATH, "utf8");
1412
+ }
1413
+ catch {
1414
+ return [];
1415
+ }
1416
+ const records = [];
1417
+ for (const line of raw.split("\n")) {
1418
+ if (!line.trim())
1419
+ continue;
1420
+ try {
1421
+ const parsed = JSON.parse(line);
1422
+ if (parsed && typeof parsed === "object" && parsed.feature_id === featureId) {
1423
+ records.push(parsed);
1424
+ }
1425
+ }
1426
+ catch {
1427
+ // skip malformed line
1428
+ }
1429
+ }
1430
+ return records;
1431
+ }
1432
+ // Merges every report for a feature into one view, most recent value per
1433
+ // field wins (not most recent record wins) -- so a status_at_30d reported
1434
+ // today doesn't get lost behind an unrelated reworked update reported
1435
+ // yesterday, and vice versa. history is still returned in full for anyone
1436
+ // who wants the raw timeline rather than just the merged snapshot.
1437
+ function latestOutcomeProxy(featureId) {
1438
+ const records = readOutcomeProxyRecords(featureId).sort((a, b) => new Date(a.timestamp).getTime() - new Date(b.timestamp).getTime());
1439
+ if (records.length === 0)
1440
+ return { merged: null, history: records };
1441
+ const merged = {};
1442
+ for (const r of records) {
1443
+ if (r.reworked !== undefined)
1444
+ merged.reworked = r.reworked;
1445
+ if (r.days_to_rework !== undefined)
1446
+ merged.days_to_rework = r.days_to_rework;
1447
+ if (r.time_to_merge_hours !== undefined)
1448
+ merged.time_to_merge_hours = r.time_to_merge_hours;
1449
+ if (r.status_at_30d !== undefined)
1450
+ merged.status_at_30d = r.status_at_30d;
1451
+ }
1452
+ return { merged, history: records };
1453
+ }
1454
+ function recordOutcomeProxy(input) {
1455
+ if (input.reworked === undefined &&
1456
+ input.days_to_rework === undefined &&
1457
+ input.time_to_merge_hours === undefined &&
1458
+ input.status_at_30d === undefined) {
1459
+ throw new Error("report_outcome_proxy requires at least one of reworked, days_to_rework, time_to_merge_hours, or status_at_30d.");
1460
+ }
1461
+ const record = {
1462
+ id: randomUUID(),
1463
+ timestamp: new Date().toISOString(),
1464
+ project_id: input.project_id,
1465
+ feature_id: input.feature_id,
1466
+ ...(input.reworked !== undefined ? { reworked: input.reworked } : {}),
1467
+ ...(input.days_to_rework !== undefined ? { days_to_rework: input.days_to_rework } : {}),
1468
+ ...(input.time_to_merge_hours !== undefined ? { time_to_merge_hours: input.time_to_merge_hours } : {}),
1469
+ ...(input.status_at_30d !== undefined ? { status_at_30d: input.status_at_30d } : {}),
1470
+ };
1471
+ appendOutcomeProxyRecord(record);
1472
+ return record;
1473
+ }
1474
+ function totalFeatureCost(projectId, featureId) {
1475
+ const verdictEntries = readLedgerEntries(projectId).filter((e) => e.feature_id === featureId);
1476
+ const buildRecords = readBuildRecords(featureId).filter((r) => !r.project_id || r.project_id === projectId);
1477
+ const total = verdictEntries.reduce((sum, e) => sum + (e.cost_usd ?? 0), 0) +
1478
+ buildRecords.reduce((sum, r) => sum + (r.cost_usd ?? 0), 0);
1479
+ const { merged, history } = latestOutcomeProxy(featureId);
1480
+ return {
1481
+ feature_id: featureId,
1482
+ verdict_entries: verdictEntries,
1483
+ build_records: buildRecords,
1484
+ total_cost_usd: Math.round(total * 10000) / 10000,
1485
+ outcome_proxy: merged,
1486
+ outcome_proxy_history: history,
1487
+ };
1488
+ }
1200
1489
  // Orchestrates the ensemble: run once, and only pay for 2 more full
1201
1490
  // pipeline passes when the single-run result landed close enough to a
1202
1491
  // verdict threshold that a single item's judgment swinging could flip
@@ -1224,15 +1513,32 @@ function aggregateMeta(passes) {
1224
1513
  tokens_used: {
1225
1514
  input: metas.reduce((sum, m) => sum + m.tokens_used.input, 0),
1226
1515
  output: metas.reduce((sum, m) => sum + m.tokens_used.output, 0),
1516
+ // Only present if every pass has it -- all passes go through the same
1517
+ // buildMeta call site in practice, so a mix would mean something else
1518
+ // changed; safer to omit than to silently sum a partial set.
1519
+ ...(metas.every((m) => m.tokens_used.input_breakdown)
1520
+ ? {
1521
+ input_breakdown: {
1522
+ fresh: metas.reduce((sum, m) => sum + (m.tokens_used.input_breakdown?.fresh ?? 0), 0),
1523
+ cache_write: metas.reduce((sum, m) => sum + (m.tokens_used.input_breakdown?.cache_write ?? 0), 0),
1524
+ cache_read: metas.reduce((sum, m) => sum + (m.tokens_used.input_breakdown?.cache_read ?? 0), 0),
1525
+ },
1526
+ }
1527
+ : {}),
1227
1528
  },
1228
1529
  estimated_cost_usd: Math.round(metas.reduce((sum, m) => sum + m.estimated_cost_usd, 0) * 10000) / 10000,
1229
1530
  };
1230
1531
  }
1231
- // Builds the LedgerEntry appended after a fresh (non-cache-hit) judgment.
1532
+ // Builds the LedgerEntry appended after a judgment -- fresh (non-cache-hit)
1533
+ // or a ledger cache hit, distinguished by opts.cacheHit/opts.costUsd (a
1534
+ // cache hit is always real $0, a fresh call carries its own
1535
+ // _meta.estimated_cost_usd; callers pass that in rather than this function
1536
+ // reaching into result._meta itself, since the cache-hit path's synthetic
1537
+ // _meta shouldn't be treated as equivalent to a real one).
1232
1538
  // checklist/checklist_source come from the result itself, not input.checklist
1233
1539
  // -- that field captures what was actually scored regardless of whether the
1234
1540
  // caller pre-supplied it or this call extracted it internally.
1235
- function buildLedgerEntry(input, projectId, result) {
1541
+ function buildLedgerEntry(input, projectId, result, opts) {
1236
1542
  const candidate = distillCandidate(result);
1237
1543
  const checklist = Array.isArray(result.requirements_checked)
1238
1544
  ? result.requirements_checked.map((r) => r.requirement).filter((r) => !!r)
@@ -1241,6 +1547,7 @@ function buildLedgerEntry(input, projectId, result) {
1241
1547
  id: randomUUID(),
1242
1548
  timestamp: new Date().toISOString(),
1243
1549
  project_id: projectId,
1550
+ feature_id: deriveFeatureId(input.component_need, projectId, opts.featureId ?? input.feature_id),
1244
1551
  component_need: input.component_need,
1245
1552
  domain: input.domain,
1246
1553
  framework: input.framework,
@@ -1252,6 +1559,8 @@ function buildLedgerEntry(input, projectId, result) {
1252
1559
  confidence: result.confidence,
1253
1560
  reason: result.reason,
1254
1561
  coverage: result.coverage ?? null,
1562
+ cost_usd: opts.costUsd,
1563
+ cache_hit: opts.cacheHit,
1255
1564
  project_conventions_snapshot: hashConventions(input.existing_stack),
1256
1565
  };
1257
1566
  }
@@ -1295,6 +1604,27 @@ async function judgeComponent(input) {
1295
1604
  estimated_cost_usd: 0,
1296
1605
  },
1297
1606
  };
1607
+ captureRecommendation({
1608
+ projectId: input.project_id,
1609
+ verdict: result.verdict,
1610
+ confidence: result.confidence,
1611
+ reason: result.reason,
1612
+ ensembleTriggered: false,
1613
+ estimatedCostUsd: 0,
1614
+ servedFromLedger: true,
1615
+ });
1616
+ // Cost-attribution build plan, 1.1: log feature_id on every ledger
1617
+ // write, cache hit included -- not just fresh judgments -- so a
1618
+ // feature's total cost rolls up correctly even when most of its later
1619
+ // calls cost $0 via this exact short-circuit. Inherits the matched
1620
+ // entry's feature_id unless this call explicitly supplies its own.
1621
+ if (input.project_id) {
1622
+ appendLedgerEntry(buildLedgerEntry(input, input.project_id, result, {
1623
+ costUsd: 0,
1624
+ cacheHit: true,
1625
+ featureId: input.feature_id ?? ledgerCacheHit.feature_id,
1626
+ }));
1627
+ }
1298
1628
  return JSON.stringify(result);
1299
1629
  }
1300
1630
  // Session cap and local logging both apply only to calls that actually
@@ -1319,7 +1649,21 @@ async function judgeComponent(input) {
1319
1649
  if (reachesApi)
1320
1650
  logCall(input, first.result);
1321
1651
  if (reachesApi && input.project_id && (first.result.reason === "scored" || first.result.reason === "no_candidates_found")) {
1322
- appendLedgerEntry(buildLedgerEntry(input, input.project_id, first.result));
1652
+ appendLedgerEntry(buildLedgerEntry(input, input.project_id, first.result, {
1653
+ costUsd: first.result._meta?.estimated_cost_usd ?? 0,
1654
+ cacheHit: false,
1655
+ }));
1656
+ }
1657
+ if (reachesApi) {
1658
+ captureRecommendation({
1659
+ projectId: input.project_id,
1660
+ verdict: first.result.verdict,
1661
+ confidence: first.result.confidence,
1662
+ reason: first.result.reason,
1663
+ ensembleTriggered: false,
1664
+ estimatedCostUsd: first.result._meta?.estimated_cost_usd ?? null,
1665
+ servedFromLedger: false,
1666
+ });
1323
1667
  }
1324
1668
  return JSON.stringify(first.result);
1325
1669
  }
@@ -1328,13 +1672,37 @@ async function judgeComponent(input) {
1328
1672
  reason: first.result.reason,
1329
1673
  coverage: first.result.coverage,
1330
1674
  }));
1331
- const [second, third] = await Promise.all([runSinglePass(input), runSinglePass(input)]);
1332
- const passes = [first, second, third].filter((p) => p.ok);
1333
- const verdicts = passes.map((p) => p.result.verdict);
1334
- const counts = new Map();
1675
+ // Adaptive escalation: run only a 2nd pass first. A binary verdict
1676
+ // (use_existing | custom_build) can only tie at 2 passes, never at 3 --
1677
+ // so we escalate to a 3rd pass ONLY on that 1/1 tie, which is exactly
1678
+ // the case that actually needs a tie-break. When the 2nd pass agrees
1679
+ // with the 1st, that agreement is itself the answer and a 3rd pass
1680
+ // would just spend real API cost confirming what's already settled.
1681
+ // This does not touch the correctness guarantee for genuine
1682
+ // disagreement -- it still always resolves via an odd-numbered
1683
+ // majority vote, same as the flat 3-run version this replaces.
1684
+ const second = await runSinglePass(input);
1685
+ let passes = [first, second].filter((p) => p.ok);
1686
+ let verdicts = passes.map((p) => p.result.verdict);
1687
+ let counts = new Map();
1335
1688
  for (const v of verdicts)
1336
1689
  counts.set(v, (counts.get(v) ?? 0) + 1);
1337
- const [majorityVerdict, majorityCount] = [...counts.entries()].sort((a, b) => b[1] - a[1])[0];
1690
+ let sortedCounts = [...counts.entries()].sort((a, b) => b[1] - a[1]);
1691
+ const isTwoWayTie = passes.length === 2 && sortedCounts.length === 2 && sortedCounts[0][1] === sortedCounts[1][1];
1692
+ if (isTwoWayTie) {
1693
+ console.error(JSON.stringify({
1694
+ diagnostic: "ensemble_tie_escalated",
1695
+ runs: verdicts,
1696
+ }));
1697
+ const third = await runSinglePass(input);
1698
+ passes = [first, second, third].filter((p) => p.ok);
1699
+ verdicts = passes.map((p) => p.result.verdict);
1700
+ counts = new Map();
1701
+ for (const v of verdicts)
1702
+ counts.set(v, (counts.get(v) ?? 0) + 1);
1703
+ sortedCounts = [...counts.entries()].sort((a, b) => b[1] - a[1]);
1704
+ }
1705
+ const [majorityVerdict, majorityCount] = sortedCounts[0];
1338
1706
  const agreement = `${majorityCount}/${passes.length}`;
1339
1707
  // Use a pass whose own verdict already matches the majority as the base
1340
1708
  // for everything else in the response (recommendation, coverage,
@@ -1379,8 +1747,20 @@ async function judgeComponent(input) {
1379
1747
  // reachesApi === true here, no guard needed.
1380
1748
  logCall(input, base);
1381
1749
  if (input.project_id && (base.reason === "scored" || base.reason === "no_candidates_found")) {
1382
- appendLedgerEntry(buildLedgerEntry(input, input.project_id, base));
1750
+ appendLedgerEntry(buildLedgerEntry(input, input.project_id, base, {
1751
+ costUsd: base._meta?.estimated_cost_usd ?? 0,
1752
+ cacheHit: false,
1753
+ }));
1383
1754
  }
1755
+ captureRecommendation({
1756
+ projectId: input.project_id,
1757
+ verdict: base.verdict,
1758
+ confidence: base.confidence,
1759
+ reason: base.reason,
1760
+ ensembleTriggered: true,
1761
+ estimatedCostUsd: base._meta?.estimated_cost_usd ?? null,
1762
+ servedFromLedger: false,
1763
+ });
1384
1764
  return JSON.stringify(base);
1385
1765
  }
1386
1766
  // The model's stated `coverage` string doesn't always match its own
@@ -1477,7 +1857,26 @@ export function enforceVerdictThreshold(parsed) {
1477
1857
  let correctConfidence;
1478
1858
  if (pct >= 80) {
1479
1859
  correctVerdict = "use_existing";
1480
- correctConfidence = "high";
1860
+ // Oversized Match overrides the coverage-only threshold -- a candidate
1861
+ // can satisfy every requirement and still be the wrong call if it's
1862
+ // disproportionate to the stated scope (see step 5's Oversized Match
1863
+ // check and the JudgmentResult.oversized_match comment). Deliberately
1864
+ // keyed off the model's own oversized_match flag, not its "confidence"
1865
+ // field -- confirmed live that the model can correctly reason through
1866
+ // an Oversized Match in oversized_match_note and still leave
1867
+ // "confidence": "high" unchanged, so that field alone can't be trusted
1868
+ // to carry this signal.
1869
+ if (parsed.oversized_match === true) {
1870
+ correctConfidence = "low";
1871
+ console.error(JSON.stringify({
1872
+ diagnostic: "oversized_match_confidence_capped",
1873
+ coverage: parsed.coverage,
1874
+ note: parsed.oversized_match_note ?? null,
1875
+ }));
1876
+ }
1877
+ else {
1878
+ correctConfidence = "high";
1879
+ }
1481
1880
  }
1482
1881
  else if (pct >= 40) {
1483
1882
  correctVerdict = "use_existing";
@@ -1798,6 +2197,39 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
1798
2197
  "instead of a fresh search+score).",
1799
2198
  inputSchema: READ_LEDGER_INPUT_SCHEMA,
1800
2199
  },
2200
+ {
2201
+ name: REPORT_BUILD_COST_TOOL_NAME,
2202
+ description: "Self-reports the end-to-end build cost for one feature -- call this " +
2203
+ "once when the build a recommend_component verdict fed into is " +
2204
+ "actually complete (shipped, abandoned, or replaced), not on every " +
2205
+ "verdict. Pattern only ever sees the cost of judging what to use; " +
2206
+ "everything past that -- the actual scaffold, install, or custom " +
2207
+ "build -- happens outside Pattern entirely, so this is the only way " +
2208
+ "that cost gets attributed back to the feature. Pass the same " +
2209
+ "feature_id you used (or that recommend_component derived) for this " +
2210
+ "feature's judgment call(s), so read_ledger's feature_id rollup can " +
2211
+ "join this record to them. This only appends a local record; it " +
2212
+ "never re-runs any judgment and never calls the Anthropic API.",
2213
+ inputSchema: REPORT_BUILD_COST_INPUT_SCHEMA,
2214
+ },
2215
+ {
2216
+ name: REPORT_OUTCOME_PROXY_TOOL_NAME,
2217
+ description: "Self-reports a value signal for one feature that is deliberately " +
2218
+ "independent of Pattern's own verdict -- never derive any of these " +
2219
+ "fields from coverage_pct, confidence, or anything else Pattern " +
2220
+ "returned; they only mean something if they could contradict the " +
2221
+ "verdict. Compute reworked/days_to_rework and time_to_merge_hours " +
2222
+ "from your own repo's real git history (e.g. `git log --follow` " +
2223
+ "against the files this feature's build touched) -- never guess " +
2224
+ "them. Report status_at_30d only once a real ~30-day-post-merge " +
2225
+ "horizon has actually passed. Safe to call more than once for the " +
2226
+ "same feature_id as more signal becomes available over time (e.g. " +
2227
+ "time_to_merge_hours right after merge, reworked on a later check, " +
2228
+ "status_at_30d at the 30-day mark) -- read_ledger's feature_id " +
2229
+ "rollup merges every report into one latest-value-per-field view. " +
2230
+ "This only appends a local record; it never calls the Anthropic API.",
2231
+ inputSchema: REPORT_OUTCOME_PROXY_INPUT_SCHEMA,
2232
+ },
1801
2233
  ],
1802
2234
  }));
1803
2235
  server.setRequestHandler(CallToolRequestSchema, async (request) => {
@@ -1811,6 +2243,9 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
1811
2243
  }
1812
2244
  catch (err) {
1813
2245
  const message = err instanceof Error ? err.message : String(err);
2246
+ if (/Anthropic API error \d+/.test(message)) {
2247
+ captureApiError({ tool: TOOL_NAME, message, projectId: args.project_id });
2248
+ }
1814
2249
  return {
1815
2250
  content: [{ type: "text", text: `Error: ${message}` }],
1816
2251
  isError: true,
@@ -1839,6 +2274,9 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
1839
2274
  }
1840
2275
  catch (err) {
1841
2276
  const message = err instanceof Error ? err.message : String(err);
2277
+ if (/Anthropic API error \d+/.test(message)) {
2278
+ captureApiError({ tool: EXTRACT_REQUIREMENTS_TOOL_NAME, message });
2279
+ }
1842
2280
  return {
1843
2281
  content: [{ type: "text", text: `Error: ${message}` }],
1844
2282
  isError: true,
@@ -1869,6 +2307,12 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
1869
2307
  if (request.params.name === READ_LEDGER_TOOL_NAME) {
1870
2308
  const args = request.params.arguments;
1871
2309
  try {
2310
+ if (args.feature_id) {
2311
+ const rollup = totalFeatureCost(args.project_id, args.feature_id);
2312
+ return {
2313
+ content: [{ type: "text", text: JSON.stringify({ project_id: args.project_id, ...rollup }) }],
2314
+ };
2315
+ }
1872
2316
  const entries = findLedgerMatches(args.project_id, args.component_need, args.limit);
1873
2317
  return {
1874
2318
  content: [{ type: "text", text: JSON.stringify({ project_id: args.project_id, entries }) }],
@@ -1882,11 +2326,52 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
1882
2326
  };
1883
2327
  }
1884
2328
  }
2329
+ if (request.params.name === REPORT_BUILD_COST_TOOL_NAME) {
2330
+ const args = request.params.arguments;
2331
+ try {
2332
+ const record = recordBuildCost(args);
2333
+ return {
2334
+ content: [{ type: "text", text: JSON.stringify({ status: "recorded", record }) }],
2335
+ };
2336
+ }
2337
+ catch (err) {
2338
+ const message = err instanceof Error ? err.message : String(err);
2339
+ return {
2340
+ content: [{ type: "text", text: `Error: ${message}` }],
2341
+ isError: true,
2342
+ };
2343
+ }
2344
+ }
2345
+ if (request.params.name === REPORT_OUTCOME_PROXY_TOOL_NAME) {
2346
+ const args = request.params.arguments;
2347
+ try {
2348
+ const record = recordOutcomeProxy(args);
2349
+ return {
2350
+ content: [{ type: "text", text: JSON.stringify({ status: "recorded", record }) }],
2351
+ };
2352
+ }
2353
+ catch (err) {
2354
+ const message = err instanceof Error ? err.message : String(err);
2355
+ return {
2356
+ content: [{ type: "text", text: `Error: ${message}` }],
2357
+ isError: true,
2358
+ };
2359
+ }
2360
+ }
1885
2361
  throw new Error(`Unknown tool: ${request.params.name}`);
1886
2362
  });
1887
2363
  async function main() {
2364
+ printTelemetryNoticeOnce();
1888
2365
  const transport = new StdioServerTransport();
1889
2366
  await server.connect(transport);
2367
+ // Best-effort telemetry drain on clean shutdown -- no-op when telemetry
2368
+ // was never enabled (see src/telemetry.ts).
2369
+ for (const signal of ["SIGINT", "SIGTERM"]) {
2370
+ process.on(signal, async () => {
2371
+ await shutdownTelemetry();
2372
+ process.exit(0);
2373
+ });
2374
+ }
1890
2375
  }
1891
2376
  // Guard exists so verification scripts (e.g. verify-ledger-boundary.mjs)
1892
2377
  // can import this module's exported pure functions (distillCandidate,
@@ -0,0 +1,222 @@
1
+ /**
2
+ * Opt-in, anonymous product telemetry.
3
+ *
4
+ * Off by default. Enabling it (PATTERN_TELEMETRY=1) answers two questions
5
+ * the product can't answer any other way without asking users directly:
6
+ *
7
+ * - Do people come back and use Pattern on a second or third project on
8
+ * their own, unprompted? (tracked via distinct project hashes seen per
9
+ * anonymous install, on every recommend_component call)
10
+ * - How often does a BYO Anthropic key actually run dry or get rate
11
+ * limited in real sessions, not just the one time it happened during
12
+ * manual testing? (tracked via captureApiError)
13
+ *
14
+ * What gets sent, when enabled: an anonymous, randomly generated install
15
+ * ID (see installId() below); a one-way SHA-256 hash of project_id,
16
+ * truncated to 16 hex chars -- never the raw project_id string; the verdict
17
+ * shape already written to the local call log (verdict, confidence,
18
+ * ensemble_triggered, estimated cost); and, on a failed Anthropic API call,
19
+ * only the HTTP status and a coarse error classification (rate_limit /
20
+ * insufficient_credit / other) -- never the request or response body.
21
+ * component_need text, requirements_checked evidence, and the API key
22
+ * itself are never sent. See SECURITY.md and README.md for the full
23
+ * disclosure and the exact opt-in instructions.
24
+ *
25
+ * Reuses Pattern's existing PostHog project (the same one the marketing
26
+ * site sends browser events to) with its public, write-only project key --
27
+ * safe to embed in a distributed package the same way that key is already
28
+ * embedded in the site's client bundle. CLI events are namespaced with a
29
+ * "pattern_cli_" event prefix and source: "cli" so they're never confused
30
+ * with website traffic in queries or dashboards.
31
+ */
32
+ import { PostHog } from "posthog-node";
33
+ import { createHash, randomUUID } from "node:crypto";
34
+ import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
35
+ import { homedir } from "node:os";
36
+ import { dirname, join } from "node:path";
37
+ // Opt-in, not opt-out -- deliberate, given who this tool is for. See
38
+ // README's telemetry section: a local-first tool aimed at developers who
39
+ // notice and care about silent tracking is exactly the audience §06 of the
40
+ // product brief already flags as sensitive to "no paper trail" trust gaps.
41
+ // Any of "1", "true", "yes" (case-insensitive) turns it on.
42
+ const TELEMETRY_ENABLED = /^(1|true|yes)$/i.test(process.env.PATTERN_TELEMETRY ?? "");
43
+ // One-time startup notice, printed to stderr -- the closest thing to an
44
+ // opt-in prompt an MCP stdio server can safely show. stdin is the JSON-RPC
45
+ // channel the client uses to talk to this process; blocking on it to read
46
+ // a y/n keypress would fight the protocol handshake instead of showing a
47
+ // dialog, so there's no safe way to do an interactive prompt here. This
48
+ // prints once ever (gated by TELEMETRY_NOTICE_PATH, not by whether this is
49
+ // a fresh install), so someone who installed Pattern before telemetry
50
+ // existed sees it exactly once on their first run after upgrading, the
51
+ // same as a brand-new install does on its first run ever. Call from
52
+ // main() at startup -- never from inside a tool call, so it can't be
53
+ // mistaken for a response to the calling agent.
54
+ export function printTelemetryNoticeOnce() {
55
+ try {
56
+ readFileSync(TELEMETRY_NOTICE_PATH, "utf8");
57
+ return; // Already shown -- never repeat.
58
+ }
59
+ catch {
60
+ // No marker yet -- fall through and show it.
61
+ }
62
+ const status = TELEMETRY_ENABLED
63
+ ? "ON, because PATTERN_TELEMETRY is set"
64
+ : "OFF (the default -- nothing is sent unless you opt in)";
65
+ console.error([
66
+ "",
67
+ "Pattern -- one-time telemetry notice (this will not print again)",
68
+ `Anonymous usage telemetry is currently ${status}.`,
69
+ "",
70
+ "When enabled, Pattern sends an anonymous per-install ID, a one-way",
71
+ "hash of project_id (never the raw string), and the same verdict",
72
+ "summary already written to ~/.pattern/calls.log (verdict,",
73
+ "confidence, reason, estimated cost). component_need, domain,",
74
+ "framework, existing_stack, and your API key are never sent.",
75
+ "Full field list: https://github.com/donaldrichard19-LVD/pattern-mcp#telemetry",
76
+ "",
77
+ "To help improve Pattern by sharing anonymous usage data, opt in:",
78
+ " PATTERN_TELEMETRY=1",
79
+ "Already on and want it off instead? Unset PATTERN_TELEMETRY (or set it to 0).",
80
+ "",
81
+ ].join("\n"));
82
+ try {
83
+ mkdirSync(dirname(TELEMETRY_NOTICE_PATH), { recursive: true });
84
+ writeFileSync(TELEMETRY_NOTICE_PATH, new Date().toISOString(), "utf8");
85
+ }
86
+ catch {
87
+ // Couldn't persist the marker -- worst case this prints again next
88
+ // run. Never blocks startup or a tool call over it.
89
+ }
90
+ }
91
+ // Public PostHog project API key (phc_...). Write-only: it can send events,
92
+ // it cannot read or query data back, so it's safe to ship in source the
93
+ // same way it's already shipped in the marketing site's client bundle.
94
+ // Override for self-hosting or testing against a different project.
95
+ const POSTHOG_KEY = process.env.PATTERN_POSTHOG_KEY ?? "phc_yUq5SpVfS9JxMm6QgFAYAfwzszAvbHQQsdN4xAqqJt3U";
96
+ const POSTHOG_HOST = process.env.PATTERN_POSTHOG_HOST ?? "https://us.i.posthog.com";
97
+ const INSTALL_ID_PATH = process.env.PATTERN_INSTALL_ID_PATH ?? join(homedir(), ".pattern", "install_id");
98
+ // Marker for the one-time startup notice below -- deliberately a separate
99
+ // file from install_id, not reused as an existence check. install_id gets
100
+ // created the moment ANY telemetry function runs (including a disabled
101
+ // no-op path in some future refactor); this marker exists purely to answer
102
+ // "has this specific human seen the notice yet," so it's written only from
103
+ // printTelemetryNoticeOnce itself.
104
+ const TELEMETRY_NOTICE_PATH = process.env.PATTERN_TELEMETRY_NOTICE_PATH ?? join(homedir(), ".pattern", "telemetry_notice_shown");
105
+ let cachedInstallId;
106
+ // Stable per-install anonymous ID, generated once and persisted locally --
107
+ // the distinct_id every telemetry event is keyed by. This is what makes
108
+ // "same install, second project" observable at all; without it every event
109
+ // would look like a brand-new anonymous user. Never derived from anything
110
+ // that identifies a person or machine (no hostname, no MAC, no username) --
111
+ // purely a random UUID with no way to reverse it to an identity.
112
+ function installId() {
113
+ if (cachedInstallId)
114
+ return cachedInstallId;
115
+ try {
116
+ cachedInstallId = readFileSync(INSTALL_ID_PATH, "utf8").trim();
117
+ if (cachedInstallId)
118
+ return cachedInstallId;
119
+ }
120
+ catch {
121
+ // No file yet -- fall through and create one.
122
+ }
123
+ cachedInstallId = randomUUID();
124
+ try {
125
+ mkdirSync(dirname(INSTALL_ID_PATH), { recursive: true });
126
+ writeFileSync(INSTALL_ID_PATH, cachedInstallId, "utf8");
127
+ }
128
+ catch {
129
+ // Couldn't persist (e.g. read-only home dir) -- still usable for this
130
+ // process's lifetime, just won't be stable across restarts. Telemetry
131
+ // is best-effort by design; this never blocks a tool call.
132
+ }
133
+ return cachedInstallId;
134
+ }
135
+ // One-way hash so a project_id string (which may be a real repo/project
136
+ // name someone doesn't want sent anywhere) never leaves the machine in
137
+ // readable form, while still letting the same project produce the same
138
+ // hash every time -- which is exactly what's needed to count distinct
139
+ // projects per install without ever seeing what those projects are named.
140
+ export function hashProjectId(projectId) {
141
+ return createHash("sha256").update(projectId).digest("hex").slice(0, 16);
142
+ }
143
+ // Classifies a thrown Anthropic API error by status code and the coarse
144
+ // shape of the error body, without ever inspecting or forwarding the body
145
+ // itself. Matches the two failure modes called out in the product brief's
146
+ // Risks section (§06): a key that's out of money, and rate limiting.
147
+ export function classifyApiError(message) {
148
+ const statusMatch = message.match(/Anthropic API error (\d+)/);
149
+ const status = statusMatch ? Number.parseInt(statusMatch[1], 10) : null;
150
+ if (status === 429)
151
+ return { type: "rate_limit", status };
152
+ if (status === 400 && /credit balance|insufficient/i.test(message)) {
153
+ return { type: "insufficient_credit", status };
154
+ }
155
+ return { type: "other", status };
156
+ }
157
+ let client;
158
+ function getClient() {
159
+ if (!TELEMETRY_ENABLED || !POSTHOG_KEY)
160
+ return undefined;
161
+ if (!client) {
162
+ client = new PostHog(POSTHOG_KEY, {
163
+ host: POSTHOG_HOST,
164
+ // Low volume, long-lived process (an MCP server, not a batch job) --
165
+ // flush promptly rather than buffering, so an event isn't silently
166
+ // lost if the server process is killed shortly after a call.
167
+ flushAt: 1,
168
+ flushInterval: 0,
169
+ });
170
+ }
171
+ return client;
172
+ }
173
+ // Fire-and-forget by design: telemetry must never be able to slow down or
174
+ // break a tool call. Every failure path here is swallowed, not surfaced --
175
+ // including "telemetry is disabled," which is the common case.
176
+ function capture(event, properties) {
177
+ const posthog = getClient();
178
+ if (!posthog)
179
+ return;
180
+ try {
181
+ posthog.capture({
182
+ distinctId: installId(),
183
+ event,
184
+ properties: { ...properties, source: "cli" },
185
+ });
186
+ }
187
+ catch {
188
+ // Never let a telemetry failure affect the tool call it's attached to.
189
+ }
190
+ }
191
+ export function captureRecommendation(args) {
192
+ capture("pattern_cli_recommend_component", {
193
+ project_hash: args.projectId ? hashProjectId(args.projectId) : null,
194
+ verdict: args.verdict ?? null,
195
+ confidence: args.confidence ?? null,
196
+ reason: args.reason ?? null,
197
+ ensemble_triggered: args.ensembleTriggered ?? false,
198
+ estimated_cost_usd: args.estimatedCostUsd ?? null,
199
+ served_from_ledger: args.servedFromLedger ?? false,
200
+ });
201
+ }
202
+ export function captureApiError(args) {
203
+ const { type, status } = classifyApiError(args.message);
204
+ capture("pattern_cli_api_error", {
205
+ tool: args.tool,
206
+ error_type: type,
207
+ status_code: status,
208
+ project_hash: args.projectId ? hashProjectId(args.projectId) : null,
209
+ });
210
+ }
211
+ // Best-effort drain on clean shutdown so the last event(s) of a session
212
+ // aren't dropped. Safe to call even when telemetry was never enabled.
213
+ export async function shutdownTelemetry() {
214
+ if (!client)
215
+ return;
216
+ try {
217
+ await client.shutdown();
218
+ }
219
+ catch {
220
+ // Ignore -- process is exiting either way.
221
+ }
222
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pattern-mcp",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "MCP tool that judges whether a UI component need should be met with an existing shadcn/ui, 21st.dev, or ReUI component or requires a custom build, using field/requirement coverage scored against real component code.",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
@@ -36,7 +36,8 @@
36
36
  "prepublishOnly": "npm run build"
37
37
  },
38
38
  "dependencies": {
39
- "@modelcontextprotocol/sdk": "^1.0.0"
39
+ "@modelcontextprotocol/sdk": "^1.0.0",
40
+ "posthog-node": "^5.51.4"
40
41
  },
41
42
  "devDependencies": {
42
43
  "@types/node": "^22.0.0",