pattern-mcp 0.1.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1207 -441
- package/dist/index.js +951 -127
- package/dist/staged/anthropic.js +110 -0
- package/dist/staged/extract.js +22 -0
- package/dist/staged/pipeline.js +103 -0
- package/dist/staged/reference.js +67 -0
- package/dist/staged/score.js +48 -0
- package/dist/staged/search.js +41 -0
- package/dist/staged/types.js +7 -0
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,54 +1,185 @@
|
|
|
1
1
|
# Pattern
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
3
|
+
[](https://github.com/donaldrichard19-LVD/pattern-mcp/actions/workflows/publish.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/pattern-mcp)
|
|
5
|
+
[](https://www.npmjs.com/package/pattern-mcp)
|
|
6
|
+
[](./LICENSE)
|
|
7
|
+
|
|
8
|
+
Pattern is an MCP server that checks a UI component need against real,
|
|
9
|
+
current evidence before your agent commits to it, so a wrong decision
|
|
10
|
+
gets caught before it's built, not after.
|
|
11
|
+
|
|
12
|
+
[Website](https://usepattern.sh) · [npm](https://www.npmjs.com/package/pattern-mcp) · [Report an issue](https://github.com/donaldrichard19-LVD/pattern-mcp/issues/new/choose)
|
|
13
|
+
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npm install pattern-mcp
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
See [Quick Start](#quick-start) below to add your Anthropic API key and connect
|
|
21
|
+
Pattern to your MCP client.
|
|
22
|
+
|
|
23
|
+
## What Pattern Does
|
|
24
|
+
|
|
25
|
+
Instead of returning a list of search results, Pattern looks at what you
|
|
26
|
+
need, checks real components against that need, and tells the agent
|
|
27
|
+
whether to:
|
|
28
|
+
|
|
29
|
+
- **Use an existing component** from shadcn/ui, 21st.dev, or ReUI
|
|
30
|
+
- **Build a custom component**, using a real product reference from
|
|
31
|
+
Mobbin and/or Figma Community
|
|
32
|
+
|
|
33
|
+
Pattern is designed for agents to use **while they are building**.
|
|
34
|
+
|
|
35
|
+
It exposes four tools:
|
|
36
|
+
|
|
37
|
+
- `recommend_component` — evaluates a UI component need and returns a
|
|
38
|
+
structured recommendation.
|
|
39
|
+
- `extract_requirements` — runs just the requirement-extraction step on
|
|
40
|
+
its own, so you can inspect or hand-edit the checklist before
|
|
41
|
+
`recommend_component` spends its search+score budget on it.
|
|
42
|
+
- `record_component_decision` — records what the agent actually did so
|
|
43
|
+
future recommendations in the same project can take that decision into
|
|
44
|
+
account.
|
|
45
|
+
- `read_ledger` — lists past `recommend_component` judgments for a
|
|
46
|
+
`project_id`, including any that were served from the ledger cache (see
|
|
47
|
+
[Per-project judgment ledger](#per-project-judgment-ledger)).
|
|
19
48
|
|
|
20
49
|
## How it works
|
|
21
50
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
For each `recommend_component` call, Pattern:
|
|
54
|
+
|
|
55
|
+
1. Checks whether the need is a simple primitive that doesn't require a
|
|
56
|
+
search.
|
|
57
|
+
2. Turns the request into a set of specific requirements, unless a
|
|
58
|
+
checklist was already supplied (see [`checklist`](#checklist)).
|
|
59
|
+
3. Searches for matching shadcn/ui, 21st.dev, and ReUI components.
|
|
60
|
+
4. Checks each candidate against the requirements using evidence from the
|
|
61
|
+
actual component.
|
|
62
|
+
5. Calculates how much of the requirement is covered.
|
|
63
|
+
6. Decides whether to use an existing component or build a custom one.
|
|
64
|
+
7. If a custom build is needed, searches Mobbin and Figma Community for
|
|
65
|
+
real product examples.
|
|
66
|
+
8. Returns the result as structured JSON the calling agent can act on.
|
|
67
|
+
|
|
68
|
+
Coverage is calculated by the server from the individual requirements it
|
|
69
|
+
checked. It does not simply trust the percentage returned by the model.
|
|
70
|
+
|
|
71
|
+
A result can also be:
|
|
72
|
+
|
|
73
|
+
- `use_existing`
|
|
74
|
+
- `custom_build`
|
|
75
|
+
- `no_candidates_found`
|
|
76
|
+
- `skip_list`
|
|
77
|
+
- `ledger_cache_hit` — served from a recent, matching prior judgment
|
|
78
|
+
instead of a fresh search+score (see
|
|
79
|
+
[Per-project judgment ledger](#per-project-judgment-ledger)).
|
|
80
|
+
|
|
81
|
+
`no_candidates_found` is kept separate from a low-coverage result. Not
|
|
82
|
+
finding a candidate is different from finding candidates that don't cover
|
|
83
|
+
the requirements.
|
|
84
|
+
|
|
85
|
+
If `project_id` is supplied, Pattern also checks for past confirmed
|
|
86
|
+
decisions on that project and factors them in as a consistency signal —
|
|
87
|
+
never a rule that overrides a genuinely better match found in the current
|
|
88
|
+
search. Separately, `project_id` also enables the judgment ledger: a
|
|
89
|
+
high-confidence prior judgment matching this exact
|
|
90
|
+
component_need/domain/framework/existing_stack, recorded recently enough,
|
|
91
|
+
can be served directly (`ledger_cache_hit`) instead of running a fresh
|
|
92
|
+
search+score. This is the one deliberate exception to "every recommendation
|
|
93
|
+
searches and scores again" — see
|
|
94
|
+
[Per-project judgment ledger](#per-project-judgment-ledger) for the exact
|
|
95
|
+
rules and why it's safe.
|
|
96
|
+
|
|
97
|
+
Every result includes `computed_at`, because coverage is a snapshot of the
|
|
98
|
+
search at that point in time, not a permanent fact. Every result also
|
|
99
|
+
includes `_meta` — the timing and token cost of that specific call (see
|
|
100
|
+
[Cost](#cost)).
|
|
101
|
+
|
|
102
|
+
### Boundary-risk checks
|
|
103
|
+
|
|
104
|
+
The same evidence can sometimes be judged slightly differently between
|
|
105
|
+
model runs. When a result is close enough to a decision threshold that it
|
|
106
|
+
could change the verdict, Pattern automatically runs the judgment two more
|
|
107
|
+
times and uses the majority result.
|
|
108
|
+
|
|
109
|
+
If the three runs disagree, Pattern returns:
|
|
110
|
+
|
|
111
|
+
```json
|
|
112
|
+
{
|
|
113
|
+
"confidence": "low",
|
|
114
|
+
"ensemble": {
|
|
115
|
+
"triggered": true,
|
|
116
|
+
"runs": ["use_existing", "custom_build", "use_existing"],
|
|
117
|
+
"agreement": "2/3"
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Results that are clearly inside a threshold don't trigger extra runs — see
|
|
123
|
+
[Cost](#cost) below for the measured impact.
|
|
124
|
+
|
|
125
|
+
### Simple primitives
|
|
126
|
+
|
|
127
|
+
These are handled locally without an API call:
|
|
128
|
+
|
|
129
|
+
| Primitive | Use it for |
|
|
130
|
+
| --- | --- |
|
|
131
|
+
| `button` | A clickable action trigger |
|
|
132
|
+
| `input` | A single-line text entry field |
|
|
133
|
+
| `checkbox` | A binary on/off toggle |
|
|
134
|
+
| `label` | A caption for a field or control |
|
|
135
|
+
| `badge` | A small status or count indicator |
|
|
136
|
+
| `spinner` | An indeterminate loading indicator |
|
|
137
|
+
| `tooltip` | A contextual hover/focus hint |
|
|
138
|
+
| `avatar` | A user or entity image, or initials |
|
|
139
|
+
| `icon` | A single glyph or symbol |
|
|
140
|
+
|
|
141
|
+
This keeps trivial requests fast and avoids unnecessary API usage.
|
|
142
|
+
|
|
143
|
+
### What powers the search
|
|
144
|
+
|
|
145
|
+
Pattern does not scrape shadcn/ui, 21st.dev, ReUI, Mobbin, or Figma Community
|
|
146
|
+
itself.
|
|
147
|
+
|
|
148
|
+
Each tool call makes one or more requests to the Anthropic Messages API,
|
|
149
|
+
using `claude-sonnet-5` by default. The server enables Anthropic's
|
|
150
|
+
`web_search` tool and provides a system prompt that defines the full
|
|
151
|
+
decision process.
|
|
152
|
+
|
|
153
|
+
That process includes:
|
|
154
|
+
|
|
155
|
+
- Skip-list checks
|
|
156
|
+
- Requirement extraction
|
|
157
|
+
- Component search
|
|
158
|
+
- Evidence-based coverage scoring
|
|
159
|
+
- Decision thresholds
|
|
160
|
+
- Mobbin and Figma Community reference searches when a custom build is
|
|
161
|
+
needed
|
|
162
|
+
|
|
163
|
+
Figma Community does not require a Figma API key. Pattern uses the same
|
|
164
|
+
web search mechanism for Figma Community as it does for the other sources.
|
|
165
|
+
|
|
166
|
+
The model returns structured JSON. Pattern then applies important checks
|
|
167
|
+
itself, including recalculating coverage and applying the decision
|
|
168
|
+
threshold.
|
|
169
|
+
|
|
170
|
+
## Quick Start
|
|
171
|
+
|
|
172
|
+
### 1. Install
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
npm install pattern-mcp
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
This installs the `pattern-mcp` command via `npx` (or your project's
|
|
179
|
+
local `node_modules/.bin`), used in the client configs below.
|
|
180
|
+
|
|
181
|
+
<details>
|
|
182
|
+
<summary>Build from source instead</summary>
|
|
52
183
|
|
|
53
184
|
```bash
|
|
54
185
|
git clone <this repo>
|
|
@@ -57,104 +188,172 @@ npm install
|
|
|
57
188
|
npm run build
|
|
58
189
|
```
|
|
59
190
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
191
|
+
Use `node /absolute/path/to/pattern-mcp/dist/index.js` as the server
|
|
192
|
+
command in place of `npx pattern-mcp` in the examples below.
|
|
193
|
+
|
|
194
|
+
</details>
|
|
195
|
+
|
|
196
|
+
### 2. Add your Anthropic API key
|
|
197
|
+
|
|
198
|
+
Pattern requires:
|
|
199
|
+
|
|
200
|
+
```
|
|
201
|
+
ANTHROPIC_API_KEY
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The API account associated with this key pays for the requests Pattern
|
|
205
|
+
makes (see [Cost](#cost) below).
|
|
206
|
+
|
|
207
|
+
You get the key from the Anthropic Console under Settings → API Keys.
|
|
208
|
+
API billing is separate from Claude.ai or Claude Code subscriptions. A
|
|
209
|
+
Claude Pro or Max subscription does not include API usage.
|
|
210
|
+
|
|
211
|
+
### Connect Pattern to your MCP client
|
|
212
|
+
|
|
213
|
+
Pattern is a standard MCP server, so it works with MCP-compatible
|
|
214
|
+
clients.
|
|
215
|
+
|
|
216
|
+
The server command is:
|
|
217
|
+
|
|
218
|
+
```
|
|
219
|
+
npx pattern-mcp
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
#### Claude Code
|
|
223
|
+
|
|
224
|
+
You can add Pattern to your project's `.mcp.json` or register it with the
|
|
225
|
+
CLI.
|
|
226
|
+
|
|
227
|
+
For the current project:
|
|
228
|
+
|
|
229
|
+
```bash
|
|
230
|
+
claude mcp add pattern \
|
|
231
|
+
-e ANTHROPIC_API_KEY=sk-ant-... \
|
|
232
|
+
-- npx pattern-mcp
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
This uses the default local scope, so the server is available to the
|
|
236
|
+
current project.
|
|
237
|
+
|
|
238
|
+
To make Pattern available across your projects:
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
claude mcp add pattern \
|
|
242
|
+
-e ANTHROPIC_API_KEY=sk-ant-... \
|
|
243
|
+
--scope user \
|
|
244
|
+
-- npx pattern-mcp
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
**Important:** put `-e`/`--env` and `--scope` before the `--`. Everything
|
|
248
|
+
after `--` is treated as the command and its arguments.
|
|
249
|
+
|
|
250
|
+
Check the connection with:
|
|
251
|
+
|
|
252
|
+
```bash
|
|
253
|
+
claude mcp list
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
You should see Pattern with a `✔ Connected` status.
|
|
257
|
+
|
|
258
|
+
`claude mcp add` stores the configuration in `~/.claude.json`. Avoid
|
|
259
|
+
`claude mcp get pattern` when possible because it can print your API key
|
|
260
|
+
in plaintext.
|
|
261
|
+
|
|
262
|
+
#### Cursor
|
|
263
|
+
|
|
264
|
+
Add Pattern to:
|
|
265
|
+
|
|
266
|
+
```
|
|
267
|
+
.cursor/mcp.json
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
#### Codex CLI
|
|
271
|
+
|
|
272
|
+
Pattern can be configured globally in:
|
|
273
|
+
|
|
274
|
+
```
|
|
275
|
+
~/.codex/config.toml
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
or at the project level in:
|
|
279
|
+
|
|
280
|
+
```
|
|
281
|
+
.codex/config.json
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
Use the MCP configuration format supported by your Codex CLI version.
|
|
285
|
+
|
|
286
|
+
#### Claude Desktop
|
|
287
|
+
|
|
288
|
+
Add Pattern through Claude Desktop's MCP settings.
|
|
289
|
+
|
|
290
|
+
The configuration looks like:
|
|
109
291
|
|
|
110
292
|
```json
|
|
111
293
|
{
|
|
112
294
|
"mcpServers": {
|
|
113
295
|
"pattern": {
|
|
114
|
-
"command": "
|
|
115
|
-
"args": ["
|
|
116
|
-
"env": {
|
|
296
|
+
"command": "npx",
|
|
297
|
+
"args": ["pattern-mcp"],
|
|
298
|
+
"env": {
|
|
299
|
+
"ANTHROPIC_API_KEY": "sk-ant-..."
|
|
300
|
+
}
|
|
117
301
|
}
|
|
118
302
|
}
|
|
119
303
|
}
|
|
120
304
|
```
|
|
121
305
|
|
|
122
|
-
Restart your MCP client
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
306
|
+
Restart your MCP client after adding Pattern.
|
|
307
|
+
|
|
308
|
+
Then ask your agent to list its available MCP tools and look for:
|
|
309
|
+
|
|
310
|
+
```
|
|
311
|
+
recommend_component
|
|
312
|
+
```
|
|
126
313
|
|
|
127
314
|
## Try it
|
|
128
315
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
316
|
+
Give your agent a specific UI need, for example:
|
|
317
|
+
|
|
318
|
+
> Use recommend_component to find me a UI component for a price breakdown
|
|
319
|
+
> showing nightly rate, cleaning fee, service fee, and taxes. I'm building
|
|
320
|
+
> an Airbnb-style booking checkout in React with Tailwind.
|
|
321
|
+
|
|
322
|
+
The agent should use the result to make the next decision:
|
|
323
|
+
|
|
324
|
+
- Install or use the recommended component, or
|
|
325
|
+
- Start a custom build using the returned requirements and product
|
|
326
|
+
references.
|
|
327
|
+
|
|
328
|
+
Pattern returns useful descriptions for both paths.
|
|
329
|
+
|
|
330
|
+
- For an existing component, `component_description` explains what the
|
|
331
|
+
component does and looks like before the agent installs it.
|
|
332
|
+
- For a custom build, `reference_description` explains what each Mobbin
|
|
333
|
+
or Figma Community reference actually shows.
|
|
334
|
+
|
|
335
|
+
These descriptions are grounded in what Pattern found during the search
|
|
336
|
+
rather than generic descriptions.
|
|
337
|
+
|
|
338
|
+
## Validation examples
|
|
339
|
+
|
|
340
|
+
Pattern's validation suite uses five UI needs from an Airbnb-style rental
|
|
341
|
+
marketplace:
|
|
342
|
+
|
|
343
|
+
- Price breakdown with fees and taxes
|
|
344
|
+
- Cancellation policy display
|
|
345
|
+
- Host earnings dashboard
|
|
346
|
+
- Property image gallery
|
|
347
|
+
- Host-guest messaging inbox
|
|
348
|
+
|
|
349
|
+
Together, these cover different outcomes, including clear matches,
|
|
350
|
+
false-positive-prone searches, no candidates, and decisions close to the
|
|
351
|
+
threshold.
|
|
154
352
|
|
|
155
353
|
## Tool: `recommend_component`
|
|
156
354
|
|
|
157
|
-
|
|
355
|
+
### Input
|
|
356
|
+
|
|
158
357
|
```json
|
|
159
358
|
{
|
|
160
359
|
"component_need": "price breakdown with fees and taxes",
|
|
@@ -164,137 +363,250 @@ messaging inbox — all in the same Airbnb-style rental marketplace domain.
|
|
|
164
363
|
"project_id": "my-booking-app"
|
|
165
364
|
}
|
|
166
365
|
```
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
`
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
366
|
+
|
|
367
|
+
`component_need` should describe the actual UI you need, not just a
|
|
368
|
+
category.
|
|
369
|
+
|
|
370
|
+
Good: `price breakdown with fees and taxes`
|
|
371
|
+
Too vague: `pricing`
|
|
372
|
+
|
|
373
|
+
Vague requests can produce misleading matches. For example, a generic
|
|
374
|
+
SaaS pricing table may look like a match for "pricing" even though it
|
|
375
|
+
doesn't work for a booking checkout.
|
|
376
|
+
|
|
377
|
+
#### `project_id`
|
|
378
|
+
|
|
379
|
+
`project_id` is optional.
|
|
380
|
+
|
|
381
|
+
When provided, Pattern can use decisions previously recorded for the
|
|
382
|
+
same project (see [Per-project decision memory](#per-project-decision-memory))
|
|
383
|
+
as a consistency signal.
|
|
384
|
+
|
|
385
|
+
A previous decision can help the model stay consistent with similar UI
|
|
386
|
+
decisions, but it cannot override a better match found in the current
|
|
387
|
+
search.
|
|
388
|
+
|
|
389
|
+
Pattern still searches and scores every request from scratch. Past
|
|
390
|
+
decisions never cause a search to be skipped.
|
|
391
|
+
|
|
392
|
+
If you leave out `project_id`, Pattern does not use project memory.
|
|
393
|
+
|
|
394
|
+
#### `checklist`
|
|
395
|
+
|
|
396
|
+
`checklist` is optional -- an array of requirement strings.
|
|
397
|
+
|
|
398
|
+
When provided, `recommend_component` skips its own internal requirement
|
|
399
|
+
extraction entirely and scores coverage against exactly the items you
|
|
400
|
+
passed, instead of extracting its own checklist. Search and scoring still
|
|
401
|
+
run fresh every call; only the extraction step is skipped.
|
|
402
|
+
|
|
403
|
+
This is meant to be used together with [`extract_requirements`](#tool-extract_requirements):
|
|
404
|
+
call `extract_requirements` first, inspect (or hand-edit) the checklist it
|
|
405
|
+
returns, then pass that checklist here. That gives you a chance to catch a
|
|
406
|
+
misread requirement before Pattern spends its search+score budget.
|
|
407
|
+
|
|
408
|
+
Leave `checklist` out to keep today's default behavior: `recommend_component`
|
|
409
|
+
extracts its own checklist internally, exactly as before this option
|
|
410
|
+
existed.
|
|
411
|
+
|
|
412
|
+
**Is the checklist actually skipped, not just re-derived?** Checked, not
|
|
413
|
+
assumed. `breakdown_ms.extract` for a `checklist`-provided call is smaller
|
|
414
|
+
than the default path's, but not near-zero -- which raised the question of
|
|
415
|
+
whether the model is still doing some of the extraction work in that
|
|
416
|
+
window rather than treating the checklist as fixed input. Reading the
|
|
417
|
+
model's actual reasoning (via `thinking` with `display: "summarized"`,
|
|
418
|
+
5 runs: 3 with `checklist` provided, 2 default) answered it: the
|
|
419
|
+
`checklist`-provided runs' pre-search reasoning was a short, generic
|
|
420
|
+
"search shadcn/ui and 21st.dev" thought with no mention of the checklist's
|
|
421
|
+
content, e.g. *"I should look for existing image gallery component options
|
|
422
|
+
on shadcn/ui and 21st.dev"* -- consistently ~3-4 seconds. The default
|
|
423
|
+
runs' reasoning, by contrast, explicitly enumerated and derived the
|
|
424
|
+
checklist items (*"...mapping out the checklist: a photo grid with hero
|
|
425
|
+
and thumbnails... a full-screen lightbox with next/prev navigation,
|
|
426
|
+
keyboard support..."*) and took roughly 2x longer (~7-8 seconds). The
|
|
427
|
+
remaining time in the `checklist`-provided path is baseline model latency
|
|
428
|
+
before it decides to search, not re-extraction -- it doesn't scale with or
|
|
429
|
+
reference the checklist's content.
|
|
430
|
+
|
|
431
|
+
### Output
|
|
432
|
+
|
|
186
433
|
```json
|
|
187
434
|
{
|
|
188
435
|
"verdict": "use_existing | custom_build",
|
|
189
436
|
"confidence": "high | medium | low",
|
|
190
437
|
"reason": "scored | no_candidates_found | skip_list",
|
|
191
438
|
"computed_at": "2026-08-23",
|
|
192
|
-
"requirements_checked": [
|
|
439
|
+
"requirements_checked": [
|
|
440
|
+
{
|
|
441
|
+
"requirement": "...",
|
|
442
|
+
"met": true,
|
|
443
|
+
"evidence": "..."
|
|
444
|
+
}
|
|
445
|
+
],
|
|
193
446
|
"coverage": "5/7 (71%)",
|
|
194
447
|
"recommendation": {
|
|
195
|
-
"source": "21st.dev | shadcn | null",
|
|
448
|
+
"source": "21st.dev | shadcn | reui | null",
|
|
196
449
|
"install_command": "string | null",
|
|
197
|
-
"component_description": "string
|
|
450
|
+
"component_description": "string | null",
|
|
198
451
|
"reference": {
|
|
199
452
|
"source": "Mobbin | Figma Community",
|
|
200
453
|
"url": "...",
|
|
201
|
-
"flow_name": "...
|
|
202
|
-
"file_name": "...
|
|
454
|
+
"flow_name": "...",
|
|
455
|
+
"file_name": "...",
|
|
203
456
|
"reference_description": "...",
|
|
204
457
|
"url_type": "deep_link | entry_point"
|
|
205
458
|
}
|
|
206
459
|
},
|
|
207
|
-
"ensemble": {
|
|
208
|
-
|
|
460
|
+
"ensemble": {
|
|
461
|
+
"triggered": false
|
|
462
|
+
},
|
|
463
|
+
"checklist_source": "extracted | provided",
|
|
464
|
+
"_meta": {
|
|
465
|
+
"total_ms": 41516,
|
|
466
|
+
"breakdown_ms": { "extract": 5006, "search": 3114, "score": 33396 },
|
|
467
|
+
"tokens_used": { "input": 8400, "output": 620 },
|
|
468
|
+
"estimated_cost_usd": 0.14
|
|
469
|
+
}
|
|
209
470
|
}
|
|
210
471
|
```
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
`
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
472
|
+
|
|
473
|
+
The `past_decision_signal` field is included only when there is a
|
|
474
|
+
relevant previous decision for the supplied `project_id`.
|
|
475
|
+
|
|
476
|
+
`checklist_source` is always present: `"extracted"` when Pattern derived
|
|
477
|
+
the checklist itself (the default, unchanged behavior), `"provided"` when
|
|
478
|
+
you passed one in via `checklist`.
|
|
479
|
+
|
|
480
|
+
`_meta` is always present. See [Cost](#cost) for what each field means,
|
|
481
|
+
how `breakdown_ms` is measured, and what it means when the ensemble
|
|
482
|
+
triggers.
|
|
483
|
+
|
|
484
|
+
### Reference links
|
|
485
|
+
|
|
486
|
+
When Pattern recommends a custom build, it may return references from
|
|
487
|
+
Mobbin, Figma Community, or both.
|
|
488
|
+
|
|
489
|
+
The `reference` field can be:
|
|
490
|
+
|
|
491
|
+
- An array when both sources returned useful results.
|
|
492
|
+
- A single object when only one source returned a useful result.
|
|
493
|
+
- `null` when neither source produced a grounded reference.
|
|
494
|
+
|
|
495
|
+
#### Deep links vs. entry points
|
|
496
|
+
|
|
497
|
+
Pattern tells you whether a reference URL points directly to the
|
|
498
|
+
identified screen or flow.
|
|
499
|
+
|
|
500
|
+
`"url_type": "deep_link"` means Pattern verified that the URL points to
|
|
501
|
+
the specific reference.
|
|
502
|
+
|
|
503
|
+
`"url_type": "entry_point"` means the URL is a search or browse page. The
|
|
504
|
+
agent may need to find the specific screen or flow from there.
|
|
505
|
+
|
|
506
|
+
For Mobbin, Pattern fetches the search result page and looks for a more
|
|
507
|
+
specific link to the screen or flow it identified.
|
|
508
|
+
|
|
509
|
+
For Figma Community, URLs containing `/community/file/` are already
|
|
510
|
+
specific to a file and are treated as deep links. Other Figma URLs are
|
|
511
|
+
checked like Mobbin URLs.
|
|
512
|
+
|
|
513
|
+
Pattern never invents a URL. If it cannot verify a specific link, it
|
|
514
|
+
keeps the real search result URL and clearly identifies it as an entry
|
|
515
|
+
point.
|
|
516
|
+
|
|
517
|
+
### Installation commands are not trusted
|
|
518
|
+
|
|
519
|
+
The `install_command` comes from search results. It is not verified
|
|
520
|
+
against a package registry, and Pattern does not execute it.
|
|
521
|
+
|
|
522
|
+
The calling agent should:
|
|
523
|
+
|
|
524
|
+
1. Show the command to the user.
|
|
525
|
+
2. Get confirmation.
|
|
526
|
+
3. Run it only after confirmation.
|
|
527
|
+
|
|
528
|
+
See [SECURITY.md](./SECURITY.md) for more details.
|
|
529
|
+
|
|
530
|
+
## Tool: `extract_requirements`
|
|
531
|
+
|
|
532
|
+
Runs only the requirement-extraction step `recommend_component` normally
|
|
533
|
+
does internally, and returns just the checklist -- no search, no scoring,
|
|
534
|
+
no verdict.
|
|
535
|
+
|
|
536
|
+
This is an opt-in, two-call pattern for agents that support tool search or
|
|
537
|
+
code-mode style tool use: call `extract_requirements` first, inspect (or
|
|
538
|
+
hand-edit) the checklist it returns, then pass that checklist to
|
|
539
|
+
`recommend_component`'s optional `checklist` input to score against it
|
|
540
|
+
directly, skipping `recommend_component`'s own internal extraction.
|
|
541
|
+
|
|
542
|
+
The single-call default -- just calling `recommend_component` with no
|
|
543
|
+
`checklist` -- is unchanged and is still the recommended path for most
|
|
544
|
+
callers. Reach for `extract_requirements` when you specifically want to
|
|
545
|
+
catch a misread requirement before Pattern spends its search+score budget,
|
|
546
|
+
not as a routine first step.
|
|
547
|
+
|
|
548
|
+
### Input
|
|
549
|
+
|
|
550
|
+
```json
|
|
551
|
+
{
|
|
552
|
+
"component_need": "image gallery for a property listing",
|
|
553
|
+
"domain": "Airbnb-style rental marketplace"
|
|
554
|
+
}
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
Same fields, same meaning, as `recommend_component`'s `component_need` and
|
|
558
|
+
`domain`. There is no `framework` input here -- extraction is grounded in
|
|
559
|
+
the domain, not the framework, so `framework` doesn't affect the checklist
|
|
560
|
+
in `recommend_component` either.
|
|
561
|
+
|
|
562
|
+
### Output
|
|
563
|
+
|
|
564
|
+
```json
|
|
565
|
+
{
|
|
566
|
+
"checklist": ["...", "...", "..."],
|
|
567
|
+
"extraction_confidence": "high | medium | low",
|
|
568
|
+
"_meta": {
|
|
569
|
+
"total_ms": 6798,
|
|
570
|
+
"breakdown_ms": { "extract": 6798, "search": 0, "score": 0 },
|
|
571
|
+
"tokens_used": { "input": 275, "output": 302 },
|
|
572
|
+
"estimated_cost_usd": 0.0036
|
|
573
|
+
}
|
|
574
|
+
}
|
|
575
|
+
```
|
|
576
|
+
|
|
577
|
+
Typical latency is a few seconds -- one small API call with no tools
|
|
578
|
+
declared, versus `recommend_component`'s full search+score pipeline.
|
|
579
|
+
|
|
580
|
+
**`extraction_confidence` is a placeholder heuristic, not a validated
|
|
581
|
+
signal.** It's currently derived from how specific `component_need` is
|
|
582
|
+
(word count) -- the same "vague category name" problem the rest of this
|
|
583
|
+
README warns about elsewhere. It is not based on any measured correlation
|
|
584
|
+
with actual extraction quality. Treat `"low"` as a prompt to reread your
|
|
585
|
+
`component_need`, not as a calibrated confidence score. This is flagged
|
|
586
|
+
here as a known gap, to revisit once there's real usage data to base a
|
|
587
|
+
better signal on.
|
|
588
|
+
|
|
589
|
+
Trivial primitives (see [Simple primitives](#simple-primitives)) return an
|
|
590
|
+
empty `checklist` with `extraction_confidence: "high"` and no API call, the
|
|
591
|
+
same local skip-list short-circuit `recommend_component` uses.
|
|
287
592
|
|
|
288
593
|
## Tool: `record_component_decision`
|
|
289
594
|
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
595
|
+
Use this tool after the agent has actually acted on a component
|
|
596
|
+
decision.
|
|
597
|
+
|
|
598
|
+
For example, call it after:
|
|
599
|
+
|
|
600
|
+
- Installing an existing component
|
|
601
|
+
- Completing a custom build
|
|
602
|
+
|
|
603
|
+
Do not call it for every recommendation.
|
|
604
|
+
|
|
605
|
+
The tool only saves the decision. It does not run a judgment or make an
|
|
606
|
+
Anthropic API call.
|
|
607
|
+
|
|
608
|
+
### Input
|
|
296
609
|
|
|
297
|
-
**Input:**
|
|
298
610
|
```json
|
|
299
611
|
{
|
|
300
612
|
"project_id": "my-booking-app",
|
|
@@ -302,35 +614,176 @@ effectively free and instant.
|
|
|
302
614
|
"domain": "Airbnb-style rental marketplace",
|
|
303
615
|
"action": "custom_built",
|
|
304
616
|
"source": "custom",
|
|
305
|
-
"timestamp": "2026-08-25T14:32:00.000Z"
|
|
617
|
+
"timestamp": "2026-08-25T14:32:00.000Z",
|
|
618
|
+
"time_saved_minutes": 25
|
|
306
619
|
}
|
|
307
620
|
```
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
- `
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
- `
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
621
|
+
|
|
622
|
+
- `project_id` is required and should be stable. A project directory
|
|
623
|
+
path or project name works well.
|
|
624
|
+
- `action` must be `"installed"` or `"custom_built"`.
|
|
625
|
+
- `source` can be `"shadcn"`, `"21st.dev"`, `"reui"`, or `"custom"`.
|
|
626
|
+
- `timestamp` is optional. If omitted, Pattern uses the current time.
|
|
627
|
+
- `time_saved_minutes` is optional -- the calling agent's own estimate,
|
|
628
|
+
in minutes, of how much time this decision saved by having Pattern's
|
|
629
|
+
verdict instead of researching candidates and judging fit from scratch.
|
|
630
|
+
This is entirely self-reported. Pattern has no way to measure a
|
|
631
|
+
counterfactual ("how long would this have taken without Pattern?"), so
|
|
632
|
+
unlike `_meta` (Pattern's own real cost/latency for the call that
|
|
633
|
+
produced the verdict), this number is never computed or verified --
|
|
634
|
+
it's just recorded as-given. Omit it rather than guess a number to fill
|
|
635
|
+
the field.
|
|
636
|
+
|
|
637
|
+
### Output
|
|
638
|
+
|
|
321
639
|
```json
|
|
322
|
-
{
|
|
640
|
+
{
|
|
641
|
+
"status": "recorded",
|
|
642
|
+
"project_id": "my-booking-app",
|
|
643
|
+
"entry": { "..." }
|
|
644
|
+
}
|
|
323
645
|
```
|
|
324
646
|
|
|
647
|
+
## Tool: `read_ledger`
|
|
648
|
+
|
|
649
|
+
Lists past `recommend_component` judgments for a `project_id` -- every
|
|
650
|
+
call that reached the API and produced a verdict, not just ones explicitly
|
|
651
|
+
confirmed via `record_component_decision`. Useful for auditing what
|
|
652
|
+
Pattern has already judged for a project, or for understanding why a call
|
|
653
|
+
came back with `served_from_ledger: true`.
|
|
654
|
+
|
|
655
|
+
### Input
|
|
656
|
+
|
|
657
|
+
```json
|
|
658
|
+
{
|
|
659
|
+
"project_id": "my-booking-app",
|
|
660
|
+
"component_need": "cancellation",
|
|
661
|
+
"limit": 10
|
|
662
|
+
}
|
|
663
|
+
```
|
|
664
|
+
|
|
665
|
+
- `project_id` is required.
|
|
666
|
+
- `component_need` is optional -- a simple keyword filter (substring
|
|
667
|
+
match, no embeddings) against stored entries' `component_need`. Omit to
|
|
668
|
+
list everything for the project.
|
|
669
|
+
- `limit` is optional, defaults to 20. Most recent entries first.
|
|
670
|
+
|
|
671
|
+
### Output
|
|
672
|
+
|
|
673
|
+
```json
|
|
674
|
+
{
|
|
675
|
+
"project_id": "my-booking-app",
|
|
676
|
+
"entries": [
|
|
677
|
+
{
|
|
678
|
+
"id": "a1b2c3d4-...",
|
|
679
|
+
"timestamp": "2026-08-29T19:50:47.073Z",
|
|
680
|
+
"project_id": "my-booking-app",
|
|
681
|
+
"component_need": "cancellation policy display with refund tiers by date",
|
|
682
|
+
"domain": "Airbnb-style rental marketplace",
|
|
683
|
+
"framework": "React + Tailwind",
|
|
684
|
+
"checklist": ["...", "..."],
|
|
685
|
+
"checklist_source": "extracted",
|
|
686
|
+
"candidates_evaluated": [
|
|
687
|
+
{ "source": "ReUI (reui.io)", "name": "Timeline", "url": "https://reui.io/components/timeline", "coverage_pct": 62.5 }
|
|
688
|
+
],
|
|
689
|
+
"verdict": "use_existing",
|
|
690
|
+
"chosen_candidate": "Timeline",
|
|
691
|
+
"confidence": "low",
|
|
692
|
+
"reason": "scored",
|
|
693
|
+
"coverage": "5/8 (62.5%)",
|
|
694
|
+
"project_conventions_snapshot": "9f3a1c7e2b0d4f5a"
|
|
695
|
+
}
|
|
696
|
+
]
|
|
697
|
+
}
|
|
698
|
+
```
|
|
699
|
+
|
|
700
|
+
Each entry holds only distilled fields -- `candidates_evaluated` never
|
|
701
|
+
contains raw HTML, full prop tables, or the per-requirement evidence text
|
|
702
|
+
`recommend_component` itself returns. See
|
|
703
|
+
[Data minimization](#data-minimization) below.
|
|
704
|
+
|
|
705
|
+
## Per-project judgment ledger
|
|
706
|
+
|
|
707
|
+
Distinct from [per-project decision memory](#per-project-decision-memory)
|
|
708
|
+
below -- that file only gains an entry when `record_component_decision` is
|
|
709
|
+
explicitly called. The ledger instead gains one entry automatically for
|
|
710
|
+
**every** `recommend_component` call that reaches the API with a
|
|
711
|
+
`project_id` and lands on reason `"scored"` or `"no_candidates_found"`.
|
|
712
|
+
|
|
713
|
+
Pattern stores it locally in:
|
|
714
|
+
|
|
715
|
+
```
|
|
716
|
+
~/.pattern/ledger.jsonl
|
|
717
|
+
```
|
|
718
|
+
|
|
719
|
+
Change the location with `PATTERN_LEDGER_PATH`. One JSON object per line
|
|
720
|
+
(append-only, JSONL).
|
|
721
|
+
|
|
722
|
+
### The cache-hit exception
|
|
723
|
+
|
|
724
|
+
Every other part of Pattern scores fresh every time (see
|
|
725
|
+
[No caching, by design](#no-caching-by-design)). The ledger is the one
|
|
726
|
+
deliberate exception: a later `recommend_component` call with a matching
|
|
727
|
+
`project_id` **can** be served directly from a prior entry, skipping
|
|
728
|
+
search+score entirely, when **all** of the following hold:
|
|
729
|
+
|
|
730
|
+
- `component_need` matches exactly (case-insensitive).
|
|
731
|
+
- `domain` and `framework` match exactly.
|
|
732
|
+
- `existing_stack` hashes to the same value as the stored entry's
|
|
733
|
+
(both omitted counts as a match).
|
|
734
|
+
- The stored entry's `confidence` is `"high"`.
|
|
735
|
+
- The stored entry's `reason` is `"scored"` or `"no_candidates_found"`.
|
|
736
|
+
- The stored entry is no older than `PATTERN_LEDGER_TTL_DAYS` (default
|
|
737
|
+
**30** days, configurable).
|
|
738
|
+
|
|
739
|
+
When served this way, the response has `reason: "ledger_cache_hit"`,
|
|
740
|
+
`served_from_ledger: true`, `ledger_entry_id`, and
|
|
741
|
+
`original_verdict_timestamp` -- so nothing is ever silently passed off as
|
|
742
|
+
freshly verified. `_meta.estimated_cost_usd` and `tokens_used` are
|
|
743
|
+
genuinely `0`: no API call happened. `requirements_checked` is `null` on
|
|
744
|
+
this path -- the ledger never stores per-requirement evidence text (see
|
|
745
|
+
[Data minimization](#data-minimization)), so a cache hit can only replay
|
|
746
|
+
the verdict/confidence/coverage/chosen-candidate, not the original
|
|
747
|
+
per-requirement reasoning.
|
|
748
|
+
|
|
749
|
+
Any mismatch on the criteria above -- a different `domain`, a changed
|
|
750
|
+
`existing_stack`, an entry that's gone stale, or one that wasn't
|
|
751
|
+
high-confidence -- falls through to a normal, fresh search+score call.
|
|
752
|
+
|
|
753
|
+
### Turning the cache-hit exception off
|
|
754
|
+
|
|
755
|
+
Set `PATTERN_NO_LEDGER_CACHE_HIT` (any truthy value) to restore
|
|
756
|
+
"every `recommend_component` call always scores fresh" without removing
|
|
757
|
+
any ledger code. This disables only the cache-hit short-circuit --
|
|
758
|
+
entries are still written to `ledger.jsonl` and `read_ledger` still works
|
|
759
|
+
either way, so the audit trail keeps growing even with the switch on.
|
|
760
|
+
Unset the variable to re-enable cache hits again at any time.
|
|
761
|
+
|
|
762
|
+
### Data minimization
|
|
763
|
+
|
|
764
|
+
Nothing written to the ledger ever contains raw search/fetch content.
|
|
765
|
+
Every candidate is reduced to exactly four fields before it's written --
|
|
766
|
+
`source`, `name`, `url`, `coverage_pct` -- enforced at the type level
|
|
767
|
+
(`assertDistilledCandidateShape` in `src/index.ts`), not just by
|
|
768
|
+
convention: a raw or extended object throws rather than silently
|
|
769
|
+
persisting. Run `node scripts/verify-ledger-boundary.mjs` (after
|
|
770
|
+
`npm run build`) to check this boundary directly.
|
|
771
|
+
|
|
325
772
|
## Per-project decision memory
|
|
326
773
|
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
774
|
+
Pattern stores confirmed decisions locally in:
|
|
775
|
+
|
|
776
|
+
```
|
|
777
|
+
~/.pattern/memory.json
|
|
778
|
+
```
|
|
779
|
+
|
|
780
|
+
You can change the location with:
|
|
781
|
+
|
|
782
|
+
```
|
|
783
|
+
PATTERN_MEMORY_PATH
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
The file is organized by project:
|
|
334
787
|
|
|
335
788
|
```json
|
|
336
789
|
{
|
|
@@ -340,204 +793,517 @@ that's the key):
|
|
|
340
793
|
"domain": "Airbnb-style rental marketplace",
|
|
341
794
|
"action": "custom_built",
|
|
342
795
|
"source": "custom",
|
|
343
|
-
"timestamp": "2026-08-25T14:32:00.000Z"
|
|
796
|
+
"timestamp": "2026-08-25T14:32:00.000Z",
|
|
797
|
+
"time_saved_minutes": 25
|
|
344
798
|
}
|
|
345
799
|
]
|
|
346
800
|
}
|
|
347
801
|
```
|
|
348
802
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
[
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
803
|
+
`time_saved_minutes` is omitted from an entry entirely when the calling
|
|
804
|
+
agent didn't provide one -- it's never backfilled or estimated by Pattern.
|
|
805
|
+
|
|
806
|
+
Each project keeps its 50 most recent decisions. Older entries are
|
|
807
|
+
removed as new ones are added.
|
|
808
|
+
|
|
809
|
+
Only decisions explicitly recorded through `record_component_decision`
|
|
810
|
+
are saved. Pattern does not automatically save recommendations.
|
|
811
|
+
|
|
812
|
+
If an agent ignores or changes a recommendation, nothing is recorded
|
|
813
|
+
unless the agent explicitly calls `record_component_decision` with what
|
|
814
|
+
it actually did.
|
|
815
|
+
|
|
816
|
+
The memory file is local plaintext. Pattern does not send it anywhere.
|
|
817
|
+
|
|
818
|
+
`component_need` and `domain` are stored in this file, so avoid putting
|
|
819
|
+
sensitive information in them. See [SECURITY.md](./SECURITY.md).
|
|
820
|
+
|
|
821
|
+
A failure to write the decision file is returned as an error from
|
|
822
|
+
`record_component_decision`.
|
|
823
|
+
|
|
824
|
+
**No caching, by design.** Project memory (this file, `memory.json`) does
|
|
825
|
+
not cache recommendations. A previous decision is only additional context
|
|
826
|
+
for a new judgment. This is unrelated to the
|
|
827
|
+
[judgment ledger](#per-project-judgment-ledger)'s bounded cache-hit
|
|
828
|
+
exception, which lives in a separate file (`ledger.jsonl`) and is always
|
|
829
|
+
flagged (`served_from_ledger: true`) when it happens — see
|
|
830
|
+
[Known limitations](#known-limitations) for more.
|
|
831
|
+
|
|
832
|
+
## Security and privacy
|
|
833
|
+
|
|
834
|
+
Pattern uses the Anthropic API and web search to make its
|
|
835
|
+
recommendations.
|
|
836
|
+
|
|
837
|
+
Local project memory and the local call log are stored on the machine
|
|
838
|
+
running Pattern. They are not sent anywhere by Pattern itself.
|
|
839
|
+
|
|
840
|
+
Review [SECURITY.md](./SECURITY.md) before putting sensitive information
|
|
841
|
+
into fields such as `component_need`, `domain`, or project IDs.
|
|
378
842
|
|
|
379
843
|
## Cost
|
|
380
844
|
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
845
|
+
Pattern uses the Anthropic API, so `recommend_component` has a cost.
|
|
846
|
+
|
|
847
|
+
A typical single pass costs about $0.06–$0.10 with Sonnet 5 at current
|
|
848
|
+
pricing. Skip-listed primitives cost $0 because they're handled locally
|
|
849
|
+
and never reach the API. A [ledger cache hit](#the-cache-hit-exception)
|
|
850
|
+
also costs $0, for the same reason -- no API call happens.
|
|
851
|
+
|
|
852
|
+
### The `_meta` field
|
|
853
|
+
|
|
854
|
+
Every `recommend_component` and `extract_requirements` response includes
|
|
855
|
+
an internal `_meta` block reporting what that call actually spent. This
|
|
856
|
+
is not shown to the user automatically -- the calling agent has to
|
|
857
|
+
surface it, the same way it's separately instructed to show
|
|
858
|
+
`install_command` before running it (see
|
|
859
|
+
[above](#installation-commands-are-not-trusted)). Both tool descriptions
|
|
860
|
+
say so explicitly: surface `_meta.estimated_cost_usd` after the call,
|
|
861
|
+
since it's real spend against the user's own API key, not internal
|
|
862
|
+
bookkeeping.
|
|
863
|
+
|
|
864
|
+
```json
|
|
865
|
+
{
|
|
866
|
+
"total_ms": 41516,
|
|
867
|
+
"breakdown_ms": { "extract": 5006, "search": 3114, "score": 33396 },
|
|
868
|
+
"tokens_used": { "input": 8400, "output": 620 },
|
|
869
|
+
"estimated_cost_usd": 0.14,
|
|
870
|
+
"scoring_fetch": { "attempted": true, "succeeded": true, "url": "https://ui.shadcn.com/docs/components/..." }
|
|
871
|
+
}
|
|
872
|
+
```
|
|
873
|
+
|
|
874
|
+
- `total_ms` -- wall-clock time for the call.
|
|
875
|
+
- `tokens_used` -- total input tokens (fresh + cache write + cache read,
|
|
876
|
+
summed) and output tokens, read directly from the API response's own
|
|
877
|
+
usage data.
|
|
878
|
+
- `estimated_cost_usd` -- computed from `tokens_used` at Pattern's
|
|
879
|
+
configured model's current per-token rate (checked against Anthropic's
|
|
880
|
+
pricing, not assumed). This is an estimate: it doesn't account for
|
|
881
|
+
pricing changes Pattern hasn't been updated for, or any account-specific
|
|
882
|
+
discounts.
|
|
883
|
+
- `breakdown_ms` -- how `total_ms` splits across `recommend_component`'s
|
|
884
|
+
three internal phases.
|
|
885
|
+
- `scoring_fetch` -- whether step 4's single candidate-verification fetch
|
|
886
|
+
(see [Fetch-grounded scoring](#fetch-grounded-scoring-and-reference-verification)
|
|
887
|
+
below) actually happened for this response. `url` is `null` when
|
|
888
|
+
`attempted` is `false` (no real candidate to verify, e.g. `reason:
|
|
889
|
+
"no_candidates_found"` or `"skip_list"`). This is a diagnostic only --
|
|
890
|
+
Pattern never uses it to auto-correct `requirements_checked` after the
|
|
891
|
+
fact, since there's no safe fallback value for an unverified met/not-met
|
|
892
|
+
call the way there is for a reference URL.
|
|
893
|
+
|
|
894
|
+
**How `breakdown_ms` is measured, and its one real caveat.** The bundled
|
|
895
|
+
call runs extraction, search, and scoring inside a single model turn
|
|
896
|
+
(search/fetch happen server-side, not as separate requests this code
|
|
897
|
+
makes), so there's no natural place for three separate stopwatches.
|
|
898
|
+
Pattern gets a real per-phase split by streaming the response and timing
|
|
899
|
+
content-block boundaries instead: `extract` ends the moment the first
|
|
900
|
+
search call starts, and `search` ends when that first wave of search
|
|
901
|
+
calls and results finishes. This was checked against real traces (not
|
|
902
|
+
assumed) across both `use_existing` and `custom_build` cases before
|
|
903
|
+
shipping, and both boundaries land cleanly and consistently.
|
|
904
|
+
|
|
905
|
+
The one place this needs a caveat: for a `custom_build` verdict, step 6's
|
|
906
|
+
Mobbin/Figma reference search and its deep-link verification fetch happen
|
|
907
|
+
*after* the coverage-scoring reasoning that decided `custom_build` in the
|
|
908
|
+
first place -- so `breakdown_ms.score`, for those cases, covers coverage
|
|
909
|
+
scoring **and** reference-finding **and** the final write-up, not just
|
|
910
|
+
"scoring" in the narrow step-4 sense. It's still a real, measured number;
|
|
911
|
+
it's just a wider bucket for `custom_build` than for `use_existing`. This
|
|
912
|
+
is disclosed here rather than presented as a narrower number than it is.
|
|
913
|
+
|
|
914
|
+
**When the ensemble triggers** (see below), `_meta` reports the sum
|
|
915
|
+
across all reruns that actually happened -- total tokens and cost spent,
|
|
916
|
+
not the wall-clock time you waited. The three ensemble passes run with the
|
|
917
|
+
2nd and 3rd concurrent, so perceived latency is closer to ~2x one pass,
|
|
918
|
+
not the ~3x `total_ms` will show. Cost and token spend are genuinely
|
|
919
|
+
additive across reruns, which is what `_meta` is reporting there.
|
|
920
|
+
`scoring_fetch` is the one exception -- it isn't summed (a fetch either
|
|
921
|
+
happened for the specific pass whose evidence became the returned
|
|
922
|
+
`requirements_checked`, or it didn't), so it reports that winning pass's
|
|
923
|
+
own value, not an aggregate across all three.
|
|
924
|
+
|
|
925
|
+
Three things help keep the cost down without changing the decision process.
|
|
926
|
+
|
|
927
|
+
### Prompt caching
|
|
928
|
+
|
|
929
|
+
Pattern caches its system instructions using `cache_control: ephemeral`.
|
|
930
|
+
|
|
931
|
+
The instructions are the same across calls, so repeated requests don't
|
|
932
|
+
pay the full input cost for that block.
|
|
933
|
+
|
|
934
|
+
### Search limits
|
|
935
|
+
|
|
936
|
+
Pattern limits candidate discovery to 3 web searches -- one per source.
|
|
937
|
+
|
|
938
|
+
If a custom build is needed, it reserves 2 additional searches for
|
|
939
|
+
references:
|
|
940
|
+
|
|
941
|
+
- 1 for Mobbin
|
|
942
|
+
- 1 for Figma Community
|
|
943
|
+
|
|
944
|
+
shadcn/ui, 21st.dev, and ReUI are searched in the same turn rather than
|
|
945
|
+
sequentially, which reduces how much conversation context needs to be
|
|
946
|
+
sent repeatedly.
|
|
947
|
+
|
|
948
|
+
### Fetch-grounded scoring and reference verification
|
|
949
|
+
|
|
950
|
+
Pattern allows up to 3 `web_fetch` calls per pass: 1 reserved for scoring,
|
|
951
|
+
2 reserved for reference verification (1 for Mobbin, 1 for Figma
|
|
952
|
+
Community).
|
|
953
|
+
|
|
954
|
+
Before finalizing coverage, Pattern fetches the best-fitting candidate's
|
|
955
|
+
own real docs/source page once and re-checks the checklist against that
|
|
956
|
+
page, not just the search-result snippet it started with. This exists
|
|
957
|
+
because search-result descriptions can both overstate a component's real
|
|
958
|
+
capabilities and miss real ones it actually has -- both were observed in
|
|
959
|
+
testing on the same case (an invented feature claim and a missed real
|
|
960
|
+
one). If the fetch fails, or there's no confirmed URL to fetch, Pattern
|
|
961
|
+
falls back to search-only evidence and says so in the affected items.
|
|
962
|
+
|
|
963
|
+
Each result's `_meta.scoring_fetch` reports whether this fetch actually
|
|
964
|
+
happened for that response (`{ attempted, succeeded, url }`) -- it's a
|
|
965
|
+
diagnostic, not something Pattern uses to auto-correct individual
|
|
966
|
+
requirement judgments. Unlike a reference URL (which has a safe fallback:
|
|
967
|
+
the category page), there's no safe fallback for an unverified met/not-met
|
|
968
|
+
call, so nothing is silently corrected -- `scoring_fetch` just tells you
|
|
969
|
+
whether the grounding actually ran.
|
|
970
|
+
|
|
971
|
+
A fetch can read up to 15,000 content tokens. `web_fetch` has no separate
|
|
972
|
+
per-call fee; the cost comes from the content added to the model's
|
|
973
|
+
context.
|
|
974
|
+
|
|
975
|
+
### Choosing a cheaper model
|
|
976
|
+
|
|
977
|
+
You can change the model with:
|
|
978
|
+
|
|
979
|
+
```
|
|
980
|
+
PATTERN_MODEL
|
|
981
|
+
```
|
|
982
|
+
|
|
983
|
+
It defaults to:
|
|
984
|
+
|
|
985
|
+
```
|
|
986
|
+
claude-sonnet-5
|
|
987
|
+
```
|
|
988
|
+
|
|
989
|
+
You could use a cheaper model such as Haiku 4.5 without changing the code.
|
|
990
|
+
|
|
991
|
+
Before using a cheaper model in production, run the five validation cases
|
|
992
|
+
and compare its results with Sonnet's:
|
|
993
|
+
|
|
994
|
+
- Price breakdown
|
|
995
|
+
- Cancellation policy
|
|
996
|
+
- Earnings dashboard
|
|
997
|
+
- Image gallery
|
|
998
|
+
- Messaging inbox
|
|
999
|
+
|
|
1000
|
+
The cheaper model hasn't been validated yet, so these results should be
|
|
1001
|
+
treated as an open question rather than an established performance claim.
|
|
409
1002
|
|
|
410
1003
|
### Ensemble cost (boundary-risk cases only)
|
|
411
1004
|
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
1005
|
+
Pattern uses extra model calls only when a result is close enough to a
|
|
1006
|
+
decision threshold that a small change in judgment could change the
|
|
1007
|
+
verdict.
|
|
1008
|
+
|
|
1009
|
+
The requirement checklist has eight items, so coverage can only land on
|
|
1010
|
+
these values:
|
|
1011
|
+
|
|
1012
|
+
```
|
|
1013
|
+
0%
|
|
1014
|
+
12.5%
|
|
1015
|
+
25%
|
|
1016
|
+
37.5%
|
|
1017
|
+
50%
|
|
1018
|
+
62.5%
|
|
1019
|
+
75%
|
|
1020
|
+
87.5%
|
|
1021
|
+
100%
|
|
1022
|
+
```
|
|
1023
|
+
|
|
1024
|
+
The decision thresholds are 40% and 80%.
|
|
1025
|
+
|
|
1026
|
+
That means results at 37.5%, 50%, 75%, and 87.5% are the cases where
|
|
1027
|
+
changing the judgment on one requirement can flip the verdict.
|
|
1028
|
+
|
|
1029
|
+
For those cases, Pattern runs the full judgment three times and takes
|
|
1030
|
+
the majority result.
|
|
1031
|
+
|
|
1032
|
+
For example:
|
|
1033
|
+
|
|
1034
|
+
```json
|
|
1035
|
+
{
|
|
1036
|
+
"ensemble": {
|
|
1037
|
+
"triggered": true,
|
|
1038
|
+
"runs": ["use_existing", "custom_build", "use_existing"],
|
|
1039
|
+
"agreement": "2/3"
|
|
1040
|
+
}
|
|
1041
|
+
}
|
|
1042
|
+
```
|
|
1043
|
+
|
|
1044
|
+
If all three runs agree, the majority verdict is returned normally.
|
|
1045
|
+
|
|
1046
|
+
If they split 2/3, Pattern sets confidence to `"low"`. The disagreement
|
|
1047
|
+
is surfaced rather than hidden.
|
|
1048
|
+
|
|
1049
|
+
Results at 0, 12.5, 25, 62.5, and 100% stay single-pass because one
|
|
1050
|
+
changed requirement can't move them across either threshold.
|
|
1051
|
+
|
|
1052
|
+
### Measured ensemble cost
|
|
1053
|
+
|
|
1054
|
+
The ensemble doesn't mean every call costs 3x.
|
|
1055
|
+
|
|
1056
|
+
In the latest five-case validation, Pattern made 15 outer calls:
|
|
1057
|
+
|
|
1058
|
+
- 8 stayed single-pass
|
|
1059
|
+
- 7 triggered the ensemble
|
|
1060
|
+
- 21 model passes were used for those 7 ensemble calls
|
|
1061
|
+
- 29 total model calls across the test
|
|
1062
|
+
|
|
1063
|
+
That works out to about a 1.9x average multiplier across that test set.
|
|
1064
|
+
|
|
1065
|
+
The worst case is still 3x for an individual call when the ensemble is
|
|
1066
|
+
triggered.
|
|
1067
|
+
|
|
1068
|
+
### What the ensemble can and cannot solve
|
|
1069
|
+
|
|
1070
|
+
The ensemble reduces the chance that one unlucky model judgment
|
|
1071
|
+
determines the result. It doesn't eliminate uncertainty.
|
|
1072
|
+
|
|
1073
|
+
If the underlying evidence is genuinely ambiguous, three runs can still
|
|
1074
|
+
disagree.
|
|
1075
|
+
|
|
1076
|
+
For example, the image-gallery validation case continued to flip between
|
|
1077
|
+
outer runs. When that happened, the ensemble consistently reported a 2/3
|
|
1078
|
+
split with `confidence: "low"`.
|
|
1079
|
+
|
|
1080
|
+
That's expected behavior: the tool is exposing uncertainty instead of
|
|
1081
|
+
presenting an ambiguous result as certain.
|
|
454
1082
|
|
|
455
1083
|
### Session call cap
|
|
456
1084
|
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
1085
|
+
Pattern limits the number of API calls to 40 per server process by
|
|
1086
|
+
default.
|
|
1087
|
+
|
|
1088
|
+
You can change this with:
|
|
1089
|
+
|
|
1090
|
+
```
|
|
1091
|
+
PATTERN_SESSION_CAP
|
|
1092
|
+
```
|
|
1093
|
+
|
|
1094
|
+
The cap protects against runaway agents, such as an agent stuck in a
|
|
1095
|
+
retry loop or repeatedly asking for the same recommendation.
|
|
1096
|
+
|
|
1097
|
+
The 40-call default is based on the project's validation work. A
|
|
1098
|
+
realistic project with roughly 25 components would use about 25 calls
|
|
1099
|
+
for a full pass, leaving room for iteration.
|
|
1100
|
+
|
|
1101
|
+
Skip-listed primitives don't count because they never reach the API.
|
|
1102
|
+
|
|
1103
|
+
The counter lives in memory and resets when the server restarts.
|
|
1104
|
+
|
|
1105
|
+
If 40 calls is too low for your project, increase `PATTERN_SESSION_CAP`
|
|
1106
|
+
rather than repeatedly restarting the server.
|
|
470
1107
|
|
|
471
1108
|
## Local call log
|
|
472
1109
|
|
|
473
|
-
Every call
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
1110
|
+
Every API call is recorded in a local log.
|
|
1111
|
+
|
|
1112
|
+
By default:
|
|
1113
|
+
|
|
1114
|
+
```
|
|
1115
|
+
~/.pattern/calls.log
|
|
1116
|
+
```
|
|
1117
|
+
|
|
1118
|
+
You can change the location with:
|
|
1119
|
+
|
|
1120
|
+
```
|
|
1121
|
+
PATTERN_LOG_PATH
|
|
1122
|
+
```
|
|
1123
|
+
|
|
1124
|
+
The log is local. Pattern does not send it anywhere.
|
|
1125
|
+
|
|
1126
|
+
Each API call adds one JSON line, for example:
|
|
479
1127
|
|
|
480
|
-
Each line looks like:
|
|
481
1128
|
```json
|
|
482
|
-
{
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
`
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
1129
|
+
{
|
|
1130
|
+
"timestamp": "2026-08-24T21:12:43.882Z",
|
|
1131
|
+
"component_need": "cancellation policy display",
|
|
1132
|
+
"domain": "Airbnb-style rental marketplace",
|
|
1133
|
+
"framework": "React + Tailwind",
|
|
1134
|
+
"verdict": "custom_build",
|
|
1135
|
+
"confidence": "high",
|
|
1136
|
+
"reason": "scored",
|
|
1137
|
+
"coverage": "2/8 (25%)",
|
|
1138
|
+
"ensemble_triggered": false,
|
|
1139
|
+
"reference_sources_grounded": ["Mobbin", "Figma Community"],
|
|
1140
|
+
"checklist_source": "extracted",
|
|
1141
|
+
"total_ms": 44834,
|
|
1142
|
+
"estimated_cost_usd": 0.15
|
|
1143
|
+
}
|
|
1144
|
+
```
|
|
1145
|
+
|
|
1146
|
+
Additional fields appear when relevant:
|
|
1147
|
+
|
|
1148
|
+
- `ensemble_agreement` appears when the ensemble runs.
|
|
1149
|
+
- `reference_sources_grounded` appears for `custom_build` results and
|
|
1150
|
+
lists only sources that produced a grounded reference.
|
|
1151
|
+
|
|
1152
|
+
`checklist_source`, `total_ms`, and `estimated_cost_usd` mirror the
|
|
1153
|
+
call's `_meta` block (see [Cost](#cost)) -- `total_ms` and
|
|
1154
|
+
`estimated_cost_usd` are the same aggregated-across-reruns numbers when
|
|
1155
|
+
the ensemble triggers, not per-pass figures.
|
|
1156
|
+
|
|
1157
|
+
The log deliberately does not contain:
|
|
1158
|
+
|
|
1159
|
+
- The full `requirements_checked` evidence
|
|
1160
|
+
- Your Anthropic API key
|
|
1161
|
+
|
|
1162
|
+
It does contain `component_need` and `domain`, so avoid putting sensitive
|
|
1163
|
+
information in those fields. See [SECURITY.md](./SECURITY.md).
|
|
1164
|
+
|
|
1165
|
+
The log directory is created automatically.
|
|
1166
|
+
|
|
1167
|
+
If Pattern cannot write to the log because of permissions, a read-only
|
|
1168
|
+
filesystem, or a full disk, it reports the problem to stderr but does not
|
|
1169
|
+
fail the tool call.
|
|
1170
|
+
|
|
1171
|
+
### Review a log
|
|
1172
|
+
|
|
1173
|
+
You can summarize a log with:
|
|
1174
|
+
|
|
1175
|
+
```
|
|
1176
|
+
node summarize-log.js [path]
|
|
1177
|
+
```
|
|
1178
|
+
|
|
1179
|
+
If no path is provided, it uses the same default location as the server.
|
|
1180
|
+
|
|
1181
|
+
The summary includes:
|
|
1182
|
+
|
|
1183
|
+
- Verdict and confidence breakdown
|
|
1184
|
+
- Reason breakdown
|
|
1185
|
+
- Ensemble trigger and agreement rates
|
|
1186
|
+
- Reference-source grounding rates for custom builds
|
|
1187
|
+
- Component needs that were requested more than once
|
|
1188
|
+
|
|
1189
|
+
Repeated component needs can be useful to investigate alongside the
|
|
1190
|
+
[session call cap](#session-call-cap).
|
|
1191
|
+
|
|
1192
|
+
## Known limitations
|
|
1193
|
+
|
|
1194
|
+
### Model judgment can vary
|
|
1195
|
+
|
|
1196
|
+
Pattern's search results can stay the same while the model's
|
|
1197
|
+
interpretation of those results changes between runs.
|
|
1198
|
+
|
|
1199
|
+
Validation found cases where two runs found the same named components
|
|
1200
|
+
using the same search queries but judged the same evidence differently.
|
|
1201
|
+
|
|
1202
|
+
For example, the model interpreted an Export action as present in one
|
|
1203
|
+
run and absent in another.
|
|
1204
|
+
|
|
1205
|
+
This is a limitation of model-based evidence judgment, not necessarily a
|
|
1206
|
+
search or code problem.
|
|
1207
|
+
|
|
1208
|
+
The boundary-risk ensemble exists to detect and surface this uncertainty.
|
|
1209
|
+
|
|
1210
|
+
### A staged pipeline was evaluated and not adopted
|
|
1211
|
+
|
|
1212
|
+
To address the variance above, an alternative architecture was built and
|
|
1213
|
+
tested: splitting the single bundled judgment call into separate stages
|
|
1214
|
+
(extract requirements, search evidence, score coverage), on the theory
|
|
1215
|
+
that isolating each step would make results more consistent and easier
|
|
1216
|
+
to diagnose.
|
|
1217
|
+
|
|
1218
|
+
A pilot comparison (5 cases, 3 repeated runs per case, per
|
|
1219
|
+
architecture) found no consistent benefit. The staged pipeline improved
|
|
1220
|
+
consistency on one boundary-risk case but was less consistent than the
|
|
1221
|
+
bundled pipeline on another, including one run that failed outright.
|
|
1222
|
+
Net accuracy against hand-graded gold answers was statistically
|
|
1223
|
+
indistinguishable between the two architectures, and the staged
|
|
1224
|
+
pipeline cost roughly **2x** the bundled pipeline's call volume across
|
|
1225
|
+
the board, not only on the boundary-risk cases it was expected to help
|
|
1226
|
+
most.
|
|
1227
|
+
|
|
1228
|
+
Pattern ships the bundled pipeline. The staged implementation remains
|
|
1229
|
+
in the repo (`src/staged/`) as an evaluated, unshipped experiment, not
|
|
1230
|
+
a supported alternative.
|
|
1231
|
+
|
|
1232
|
+
**`extract_requirements` is not a revival of this.** It's a standalone
|
|
1233
|
+
tool for inspecting the extraction step's output before an agent commits
|
|
1234
|
+
to `recommend_component`'s search+score budget -- an opt-in visibility
|
|
1235
|
+
tool, not an internal re-architecture. `recommend_component`'s own
|
|
1236
|
+
pipeline is still fully bundled; nothing about this evaluation changed.
|
|
1237
|
+
|
|
1238
|
+
### No caching, by design
|
|
1239
|
+
|
|
1240
|
+
Every recommendation searches and scores again -- with one bounded
|
|
1241
|
+
exception (see below).
|
|
1242
|
+
|
|
1243
|
+
This means a recommendation can change as component libraries change.
|
|
1244
|
+
For example, a later shadcn/ui release can introduce a component that
|
|
1245
|
+
changes a previous `custom_build` result.
|
|
1246
|
+
|
|
1247
|
+
Do not build a second, unbounded cache of recommendations at the
|
|
1248
|
+
calling-agent layer on top of Pattern's own. If you add caching there,
|
|
1249
|
+
keep it session-scoped.
|
|
1250
|
+
|
|
1251
|
+
[Project decision memory](#per-project-decision-memory) does not change
|
|
1252
|
+
this. It provides context from previous decisions, but every
|
|
1253
|
+
`recommend_component` call still performs a fresh search and scoring
|
|
1254
|
+
pass.
|
|
1255
|
+
|
|
1256
|
+
The one deliberate exception is the
|
|
1257
|
+
[judgment ledger's cache-hit path](#the-cache-hit-exception): a later
|
|
1258
|
+
call matching an exact, recent, high-confidence prior judgment can be
|
|
1259
|
+
served without a fresh search+score. It's bounded (exact
|
|
1260
|
+
component_need/domain/framework/conventions match, a staleness TTL) and
|
|
1261
|
+
always self-identifies via `served_from_ledger: true` and
|
|
1262
|
+
`reason: "ledger_cache_hit"` -- so a calling agent that wants a guaranteed
|
|
1263
|
+
fresh check on every call should look for that flag and treat it the same
|
|
1264
|
+
as any other verdict it wants to double-check.
|
|
1265
|
+
|
|
1266
|
+
### The skip-list is still evolving
|
|
1267
|
+
|
|
1268
|
+
The primitive skip-list is a starting point and has not yet been
|
|
1269
|
+
validated against broad real-world usage.
|
|
1270
|
+
|
|
1271
|
+
Watch for two failure modes:
|
|
1272
|
+
|
|
1273
|
+
- Agents calling Pattern for things that should have been skipped.
|
|
1274
|
+
- Agents building generic UI for something that should have been on the
|
|
1275
|
+
skip-list.
|
|
1276
|
+
|
|
1277
|
+
The local call log can help identify both patterns.
|
|
1278
|
+
|
|
1279
|
+
### Pattern needs internet access
|
|
1280
|
+
|
|
1281
|
+
Pattern requires outbound access to:
|
|
1282
|
+
|
|
1283
|
+
```
|
|
1284
|
+
api.anthropic.com
|
|
1285
|
+
```
|
|
1286
|
+
|
|
1287
|
+
It also depends on whatever external sites the model's `web_search` tool
|
|
1288
|
+
can reach.
|
|
1289
|
+
|
|
1290
|
+
It will not work in an environment that blocks general outbound internet
|
|
1291
|
+
access.
|
|
1292
|
+
|
|
1293
|
+
### Requirements and coverage are judgment calls
|
|
1294
|
+
|
|
1295
|
+
Requirement extraction and evidence scoring are performed by the model.
|
|
1296
|
+
|
|
1297
|
+
Pattern adds safeguards such as:
|
|
1298
|
+
|
|
1299
|
+
- Structured requirements
|
|
1300
|
+
- Server-side coverage recalculation
|
|
1301
|
+
- Decision thresholds
|
|
1302
|
+
- Boundary-risk ensembling
|
|
1303
|
+
- Grounding checks for reference URLs
|
|
1304
|
+
|
|
1305
|
+
But the underlying interpretation of whether evidence satisfies a
|
|
1306
|
+
requirement is still model judgment.
|
|
1307
|
+
|
|
1308
|
+
When introducing Pattern into a new workflow, spot-check early results
|
|
1309
|
+
against the actual components before relying on it unattended.
|