imprint-mcp 0.6.0 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/package.json +4 -2
- package/prompts/audit-agent.md +4 -2
- package/prompts/auth-compile-agent.md +149 -158
- package/prompts/build-planning.md +14 -9
- package/prompts/compile-agent.md +46 -23
- package/prompts/live-verifier-agent.md +101 -0
- package/prompts/playbook-compilation.md +8 -1
- package/prompts/tool-candidate-detection.md +9 -29
- package/src/cli.ts +83 -1
- package/src/imprint/audit.ts +219 -20
- package/src/imprint/auth-compile-agent.ts +215 -155
- package/src/imprint/auth-compile-tools.ts +300 -118
- package/src/imprint/auth-verifier.ts +167 -180
- package/src/imprint/backend-cache.ts +18 -11
- package/src/imprint/backend-ladder.ts +225 -177
- package/src/imprint/build-plan.ts +3 -8
- package/src/imprint/cdp-browser-fetch.ts +930 -83
- package/src/imprint/claude-cli-compile.ts +46 -20
- package/src/imprint/codex-cli-compile.ts +436 -139
- package/src/imprint/compile-agent-types.ts +15 -10
- package/src/imprint/compile-agent.ts +88 -28
- package/src/imprint/compile-tools.ts +228 -39
- package/src/imprint/compile-verification.ts +187 -0
- package/src/imprint/compile.ts +71 -2
- package/src/imprint/credential-store.ts +57 -44
- package/src/imprint/emit.ts +3 -1
- package/src/imprint/freeform-redact.ts +27 -6
- package/src/imprint/live-verifier.ts +1665 -0
- package/src/imprint/llm.ts +22 -3
- package/src/imprint/login.ts +107 -25
- package/src/imprint/mcp-compile-server.ts +447 -114
- package/src/imprint/mcp-live-verifier-server.ts +359 -0
- package/src/imprint/mcp-server.ts +116 -35
- package/src/imprint/probe-backends.ts +88 -7
- package/src/imprint/recording-request.ts +36 -0
- package/src/imprint/request-capture.ts +40 -17
- package/src/imprint/runtime.ts +490 -337
- package/src/imprint/teach-state.ts +36 -2
- package/src/imprint/teach.ts +302 -146
- package/src/imprint/tool-candidates.ts +8 -10
- package/src/imprint/tool-loader.ts +12 -4
- package/src/imprint/types.ts +137 -68
package/prompts/compile-agent.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
You are the imprint compile agent. Your job is to turn a recorded browser session into a working, tested tool that returns structured output. You have tools to inspect the session, write code, run tests, and iterate until tests pass.
|
|
4
4
|
|
|
5
|
+
**Completion protocol:** never end your turn with a prose answer to the operator. Continue using the compile tools while useful investigation or implementation remains. The only valid terminal actions are the `done` tool after writing and testing artifacts, or the `give_up` tool after satisfying its narrow evidence requirements below. A recommendation to re-record is not terminal unless you actually call `give_up` with the required evidence.
|
|
6
|
+
|
|
5
7
|
## The Goal
|
|
6
8
|
|
|
7
9
|
You will produce three artifacts in the generated tool directory (`~/.imprint/<site>/<toolName>/` by default):
|
|
@@ -32,6 +34,7 @@ You will produce three artifacts in the generated tool directory (`~/.imprint/<s
|
|
|
32
34
|
Follow these steps to compile the session:
|
|
33
35
|
|
|
34
36
|
1. **Orient yourself.** Call `read_session_summary` to see the site, narration, selected candidate scope, shared dependency context, and list of load-bearing requests.
|
|
37
|
+
- If the summary contains `revisionContext`, this is a bounded revision of an existing tool. Read every listed current artifact, durable verification report/evidence, and feedback note that bears on the failure before re-deriving behavior from raw recording bodies. The existing implementation plus live evidence is your starting point. Preserve proven branches; make the smallest evidence-backed repair. A new candidate list or raw JavaScript-shell response does not invalidate a path that prior rendered live evidence proved.
|
|
35
38
|
|
|
36
39
|
If the summary includes `selectedCandidate`, compile only that candidate. Other actions in the same recording are out of scope unless they are listed as shared dependencies.
|
|
37
40
|
|
|
@@ -39,9 +42,11 @@ Follow these steps to compile the session:
|
|
|
39
42
|
|
|
40
43
|
**Inline request data.** The session summary includes `inlineData` for candidate-scoped requests (those in `selectedCandidate.requestSeqs` and `dependencySeqs`). Each entry contains the full request headers, request body, response headers, and a (possibly truncated) response body. You do NOT need to call `read_request` or `read_response_body` for these requests — the data is already available in the summary. Only use those tools if the inline response body was truncated and you need more, or for requests outside the candidate scope.
|
|
41
44
|
|
|
45
|
+
**A candidate request may be a trigger, not the result document.** A form POST can legitimately have an empty or redirect-only body while the browser immediately navigates to a document whose useful data appears after JavaScript rendering. Before concluding that a response was not captured, follow the candidate's event time range: inspect adjacent same-origin document/XHR requests and the subsequent DOM events. Candidate `requestSeqs` identify important user-controlled triggers; they are not a guarantee that the parser's final response is one of those exact sequence numbers.
|
|
46
|
+
|
|
42
47
|
**Capture hints.** The session summary may include `captureHints` — pre-built capture block suggestions derived from dual-pass analysis. When a `captureHint` exists for a server-derived value, copy its `capture` definition directly into your workflow.json's `captures` array on the indicated request, and use `${state.NAME}` in downstream requests as shown in the `usedBy` entries. This saves you from having to discover the producer response and build the capture manually.
|
|
43
48
|
|
|
44
|
-
**Parameter checklist (`likelyParams`).** When `selectedCandidate` includes a `likelyParams` array, it contains the candidate detector's analysis of which inputs the user controlled — based on the narration and request patterns. Treat this as your **parameter
|
|
49
|
+
**Parameter checklist (`likelyParams`).** When `selectedCandidate` includes a `likelyParams` array, it contains the candidate detector's analysis of which inputs the user controlled — based on the narration and request patterns. Treat this as your **parameter investigation checklist**, not an instruction to invent support. Every entry should either (a) become a grounded `${param.NAME}` in workflow.json, or (b) be deliberately omitted from `parameters` and documented in `workflow.limitations` when the recording and live evidence do not reveal a reliable encoding. Parameters that appear as `null` or `[]` in a known request field still have a grounded insertion point and should normally be templated. Never expose a parameter that is known to be ignored, broken, or implemented by guessing. Limitations are only for secondary capabilities; they cannot excuse a broken core intent. A capability omission must be closed over its dependencies: remove or narrow every public parameter description, intent claim, and related option that relies on the omitted branch. For example, omitting a branch needed by one input class while continuing to advertise that input class is not an honest limitation; either implement the branch from evidence or narrow the public tool to the working class and record that narrower capability explicitly.
|
|
45
50
|
|
|
46
51
|
**Shared modules (multi-tool runs).** If your initial context lists "Assigned shared modules" — or `read_build_plan` is available — call `read_build_plan` first. It returns prebuilt, verified helper modules under `../_shared/` that you MUST reuse instead of re-deriving their logic. For a `request-transform` module set `"requestTransformModule": "../_shared/<name>.ts"` in workflow.json; for a `parser-helper`/`types` module `import` it in `parser.ts` (e.g. `import { decode } from '../_shared/decode.ts'`). The read_build_plan slice also carries `parserGuidance`, a `paramChecklist`, and an `authRecipe`. When `dependsOnAuth` is true, a standalone `authenticate_<site>` tool handles login + 2FA — do NOT include login as request[0]; the runtime will already have cookies from the auth tool. When `dependsOnAuth` is false/absent and `authRecipe.required` is true, replicate the exact login request + `${state.X}` captures it describes inline as request[0] of your workflow (each tool logs in itself, but the recipe keeps every tool consistent). You cannot write files under `_shared/` — those modules are already built; just import them. The verifier fails this tool if an assigned module is not imported.
|
|
47
52
|
|
|
@@ -82,12 +87,13 @@ Follow these steps to compile the session:
|
|
|
82
87
|
- Replace user-variable values with `${param.NAME}` placeholders (e.g., query, date, quantity)
|
|
83
88
|
- Expose only caller-meaningful parameters. If a recorded field is an internal navigation/context/source constant (for example a page entry-point marker, tracking context, or continuation hint) and changing it does not correspond to a user-facing choice in the narration or UI, keep the recorded literal instead of advertising a caller parameter.
|
|
84
89
|
- **Vary-across-seqs fields are user input (verifier-enforced).** If a field appears multiple times in the recording's load-bearing requests with different values across seqs (e.g. `pickupDate` is `06/01/2026` in one recorded POST and `06/24/2026` in another), the recording is *proving* that field is user input. It MUST be templated as `${param.X}` (or `${state.X}` if minted by an earlier captured response, or constructed via a `requestTransformModule`). Do NOT freeze the first recording's literal value into the workflow body — the verifier diffs your body against the recorded seqs in `candidateRequestSeqs` ∪ `dependencySeqs` and rejects `done()` for every frozen-session field it finds. Constant fields (same value every seq, like `fromHomePage=true` / `country=US`) are safe to hardcode.
|
|
85
|
-
- **Use `selectedCandidate.likelyParams` as your parameter checklist** (when present).
|
|
90
|
+
- **Use `selectedCandidate.likelyParams` as your parameter investigation checklist** (when present). For every `likelyParam`, either ground and template it into a request or deliberately omit it with a durable limitation:
|
|
86
91
|
- Parameters with concrete recorded values: replace the literal value with `${param.NAME}` as usual.
|
|
87
92
|
- Parameters that are `null`, `[]`, or absent in the recorded request (filters/constraints the user toggled during recording but didn't apply in the final request state): these are **valid parameters** — add them as optional with defaults meaning "no filter applied" and template them at the correct position in the request body/URL.
|
|
88
93
|
- For positional/array-encoded bodies (JSPB, protobuf, etc.): use `sharedHelperNotes` to locate each parameter's position, and replace `null`/`[]` placeholders with `${param.NAME}`.
|
|
89
94
|
- Filter/constraint parameter defaults should use the API's "unfiltered" sentinel (typically `0`, `null`, `[]`, or empty string — infer from what the recorded request uses in that position).
|
|
90
|
-
- If a `likelyParam` genuinely has no
|
|
95
|
+
- If a secondary `likelyParam` genuinely has no evidence-backed insertion point, or live verification proves it does not work and the recording provides no safe repair, remove it from the workflow's `parameters` and add `workflow.limitations: [{ "feature": "...", "reason": "...", "omittedParameters": ["name"] }]`. Be specific about the missing recording/live evidence. Do not guess and do not leave a no-op input on the public tool. Trace the omitted feature's dependencies through the remaining schema: remove dependent parameters and narrow descriptions/intent so callers cannot select an unsupported branch indirectly. Keep a smaller honest core tool rather than a broad schema whose advertised variant returns empty or stalls.
|
|
96
|
+
- A limitation cannot hide a failure of the tool's core intent. Repair the core workflow before calling `done`.
|
|
91
97
|
- **Resolved-id params — chain the minting request, do NOT pass raw text (see `inputProvenanceHints`).** Some user-facing inputs are NOT carried in the load-bearing request as the user's text — the backend keys off a resolved opaque id (an entity/object handle, an account id, a place/geo id, a category token). The recording proves which: the request holds a value at some position that **first appears in an EARLIER response**, not in anything the user typed. `read_session_summary` surfaces these as `inputProvenanceHints` (each gives the `path`, an `example` value, the consuming `inRequestSeq`, and `mintedByResponseSeq`/`mintedByEndpoint`). For every such position:
|
|
92
98
|
- You MUST obtain the id by chaining the minting request and `capture`-ing its value, then template the captured `${state.NAME}` into that position. NEVER freeze the recorded id (it's specific to the recorded entity), and NEVER substitute the param's raw text into an id position — the backend typically ignores an unrecognized value and silently falls back to a default (an unfiltered/global result set, or a server-chosen default scope), so the call returns results that look well-formed but answer the wrong query.
|
|
93
99
|
- **`selfChain: true`** means the id is minted by the tool's OWN endpoint: the pattern is *resolve-then-refine* — issue a first request carrying the user's text (the resolver), `capture` the resolved id from its response at the recorded position, then issue the real request with `${state.NAME}` at the id position. Build this as a two-request chain (request[0] = resolve, request[1] = the load-bearing call), capturing via `extract`/`captures` exactly as for any other chained value.
|
|
@@ -114,7 +120,11 @@ Follow these steps to compile the session:
|
|
|
114
120
|
}
|
|
115
121
|
```
|
|
116
122
|
Returning a plain `string` (just the URL) still works for simple URL-signing. Use the object return when you need to build or modify the request body or headers. Do NOT invent URL query parameters as a workaround for body-encoding complexity — the server ignores unknown query params and the parameters will have no effect.
|
|
117
|
-
|
|
123
|
+
**Do not erase runtime placeholders in transforms.** If the workflow URL/body/header contains `${state.X}`, `${response[N].X}`, `${credential.X}`, or `${generated.X}`, preserve those placeholder strings or pass them through as strings. Do NOT parse them with `new URL(...).searchParams.get(...)`, `Number(...)`, `JSON.parse(...)`, date parsers, or other runtime evaluation that can turn the placeholder into `null`, `NaN`, an empty string, or the literal word `undefined`. A request transform runs before runtime substitution has real values; its job is to construct a template. If a shared helper needs `sessionId`, `buildVersion`, `csrfToken`, `nonce`, or another state value, pass the corresponding `'\${state.<name>}'` placeholder directly from the workflow contract, not by extracting it back out of an already-templated URL.
|
|
124
|
+
|
|
125
|
+
**Request-transform patch semantics.** Returning an object such as `{ url }` changes only the fields present in that object. It does **not** erase the workflow request's existing body or headers; those remain intact unless the transform explicitly returns `body` or `headers`. Returning a URL string is merely shorthand for replacing the URL. Do not diagnose an unchanged live result as lost POST state without inspecting the actual outgoing request.
|
|
126
|
+
|
|
127
|
+
**Rendered-document responses.** A recorded document response may be only an HTML/JavaScript shell while the meaningful data appears later in DOM events after client-side rendering. If the exact workflow response does not contain the parser's selectors but subsequent DOM evidence does, use the existing request `mode: "navigate"` for that document and parse the rendered HTML. Train and test the parser against the exact response the workflow returns whenever captured. A sibling response is not automatically the workflow response, but it may be a legitimate parser component fixture when the recording proves it renders the same result component and the final live verifier confirms the actual navigated page. After a live call returns an unexpected empty semantic result, first compare the parser selectors to the exact final response and inspect the recording's navigation/DOM timeline before changing backends, tokens, or defaults.
|
|
118
128
|
**Endpoint paths must be exact.** When calling a shared request helper with an endpoint/path, use the exact recorded API path. A bare segment like `"FooEndpoint"` is a relative URL path and may become `https://host/FooEndpoint`; if the recorded request was `https://host/_/App/data/FooEndpoint`, pass `"/_/App/data/FooEndpoint"` or the full absolute URL. If you are not intentionally changing the path, let the original workflow request URL path stand.
|
|
119
129
|
- **`x-api-key` is normally NOT a credential, but it is not automatically a good literal.** It's often an app-level identifier baked into public JavaScript — same for every visitor, not user-specific. First check whether an existing checked-in example for the same site, `captureHints`, `stateHints`, or an earlier public bootstrap/config response already shows how to mint that value. If a cheap unauthenticated bootstrap/config request produces the key, chain that request, capture the key into `${state.NAME}`, and reference it from the downstream API request. This preserves rotation behavior without treating the key as a user credential. Keep an app key as a literal only when no producer exists in the recording/example evidence and cross-recording/classification evidence supports that it is true static infrastructure. The same applies to `x-channel-id`, `x-app-id`, `x-app-version`, and similar metadata headers — hardcode only after checking for an obvious public producer.
|
|
120
130
|
- **NEVER use `${env.NAME}` placeholders.** The `${env.X}` syntax exists in the runtime but is reserved for operator-level configuration, not for values you can see in the recording. If a value appears in the captured request, hardcode it. If multiple candidates in the same session use different API keys for different endpoints, hardcode each one — they are endpoint-specific app constants, not secrets. The only valid placeholder types for your workflow are `${param.NAME}`, `${credential.NAME}`, `${state.NAME}`, and `${response[N].NAME}`.
|
|
@@ -180,12 +190,12 @@ Follow these steps to compile the session:
|
|
|
180
190
|
|
|
181
191
|
**Import conventions**: The runtime lives at `imprint/runtime` (resolved via a symlink at `~/.imprint/node_modules/imprint` → the repo root). Types live at `imprint/types`. During compilation, `index.ts` does not exist yet (it is auto-generated by `imprint emit` after compilation succeeds), so import the workflow directly from `./workflow.json`.
|
|
182
192
|
|
|
183
|
-
Boilerplate — use `
|
|
193
|
+
Boilerplate — use `runCapturedIntegrationCase` so the test dispatches through `runWorkflowWithLadder` (the same backend ladder the MCP server uses at runtime) while capturing the actual tool-level input and parsed result for an independent semantic verification agent. The wrapper is compile-only; it does not alter production runtime behavior. Give every invocation a stable, descriptive `caseName`. The playbook rung is intentionally excluded at this stage because `playbook.yaml` is compiled in a separate later step (`imprint compile-playbook`); the API rungs (fetch, fetch-bootstrap, cdp-replay, stealth-fetch) are available during integration-test time. The test passes as long as one rung succeeds, so a tool whose fetch path is blocked by Akamai/PerimeterX still verifies end-to-end via cdp-replay or stealth-fetch:
|
|
184
194
|
```typescript
|
|
185
195
|
import { expect, test } from 'bun:test';
|
|
186
196
|
import { dirname } from 'node:path';
|
|
187
197
|
import { fileURLToPath } from 'node:url';
|
|
188
|
-
import {
|
|
198
|
+
import { runCapturedIntegrationCase } from 'imprint/compile-verification';
|
|
189
199
|
import { loadCredentialStore } from 'imprint/runtime';
|
|
190
200
|
import type { Workflow } from 'imprint/types';
|
|
191
201
|
// index.ts is auto-generated by `imprint emit` after compilation — import workflow directly
|
|
@@ -203,7 +213,8 @@ Follow these steps to compile the session:
|
|
|
203
213
|
// load them explicitly and pass through. For unauthenticated tools,
|
|
204
214
|
// this is `undefined` and the helper proceeds without a store.
|
|
205
215
|
const credentials = (await loadCredentialStore(WORKFLOW.site)) ?? undefined;
|
|
206
|
-
const { result, usedBackend } = await
|
|
216
|
+
const { result, usedBackend } = await runCapturedIntegrationCase({
|
|
217
|
+
caseName: 'live API call returns data',
|
|
207
218
|
workflowPath: WORKFLOW_PATH,
|
|
208
219
|
params,
|
|
209
220
|
credentials,
|
|
@@ -217,17 +228,21 @@ Follow these steps to compile the session:
|
|
|
217
228
|
// a flaky test or confirming the stealth-fetch fallback worked.
|
|
218
229
|
}, 60_000);
|
|
219
230
|
```
|
|
220
|
-
The 60 s timeout is important:
|
|
231
|
+
The 60 s timeout is important: the wrapped backend ladder runs a parallel backend probe on its first call, and the cdp-replay rung needs ~33 s for a cold Chrome launch. A shorter timeout kills the test before the probe can finish, causing a false live-verification failure.
|
|
232
|
+
|
|
233
|
+
For availability, booking, event, travel, or other date-sensitive live cases,
|
|
234
|
+
derive rolling future dates at test runtime. Never reuse a recorded calendar
|
|
235
|
+
date that may be past by the time teach runs.
|
|
221
236
|
|
|
222
237
|
If both rungs fail (400, 403 across both, expired tokens), this test fails and you must fix the workflow. Common fixes: chain a session/token request first, write a `requestTransformModule` for URL signing, or use `${state.X}` captures instead of hardcoded values. If a query param changes per call (check `stateHints` for `query_param_changes_across_calls`), use `search_response_body` to find the signing function in `.js` responses and replicate it in `request-transform.ts`.
|
|
223
238
|
|
|
224
239
|
**Per-parameter coverage tests.** Beyond the baseline test above, you must write one integration test for **every parameter that has a non-default value in any captured request** (visible in `inlineData.requestBodyDecoded` or via `read_request`). Walk every recorded request, decode its body, and enumerate the set of `(paramName, nonDefaultValue)` tuples. Each tuple is a coverage unit — write a test that overrides that param and asserts a constraint on the response.
|
|
225
240
|
|
|
226
|
-
**Title each per-parameter test `param:<name> …`** — begin the title with the literal token `param:` followed by the exact parameter name (e.g. `test('param:max_price=50 constrains all results', …)`). The verifier determines coverage by which `param:<name>` tests **actually ran green against live data**, not by scanning the source: a test that is merely present but did not pass — or a whole suite that was waived by anti-bot — does NOT count as coverage. Each per-parameter test MUST call `
|
|
241
|
+
**Title each per-parameter test `param:<name> …`** — begin the title with the literal token `param:` followed by the exact parameter name (e.g. `test('param:max_price=50 constrains all results', …)`). The verifier determines coverage by which `param:<name>` tests **actually ran green against live data**, not by scanning the source: a test that is merely present but did not pass — or a whole suite that was waived by anti-bot — does NOT count as coverage. Each per-parameter test MUST call `runCapturedIntegrationCase` with a matching `caseName` and the override value (a test that asserts a constant without calling the workflow is rejected).
|
|
227
242
|
|
|
228
243
|
These tests are the only signal that each parameter actually reaches the API and affects the response. If a parameter is wired into a position the server ignores (an invented URL query param, a slot guessed wrong in a positional JSPB body), the test fails because the filtered response will look like the unfiltered one. Skipping a parameter means shipping it untested.
|
|
229
244
|
|
|
230
|
-
**ANTI-BOT SITES — minimize live calls (CRITICAL for sites like Akamai/PerimeterX/DataDome).** If the workflow's load-bearing request is a STATE-CHANGING call to a bot-defended origin — tell-tale: the recorded session carries anti-bot cookies (`_abck`, `ak_bmsc`, `bm_sv`, `datadome`, `px*`), or `fetch`/`stealth-fetch` get tarpitted/403'd — then a live `
|
|
245
|
+
**ANTI-BOT SITES — minimize live calls (CRITICAL for sites like Akamai/PerimeterX/DataDome).** If the workflow's load-bearing request is a STATE-CHANGING call to a bot-defended origin — tell-tale: the recorded session carries anti-bot cookies (`_abck`, `ak_bmsc`, `bm_sv`, `datadome`, `px*`), or `fetch`/`stealth-fetch` get tarpitted/403'd — then a live `runCapturedIntegrationCase` call PER parameter is self-defeating: the burst of state-changing calls trips the site's per-IP rate defense, which then tarpits EVERY later call **including the baseline of the next tool**, and the whole teach fails. On such sites do NOT write a live `param:<name>` test per parameter. Instead: write the ONE live **baseline** test (it proves the workflow produces real data through the trusted `fetch-bootstrap` rung), and for each non-producer-derived parameter do the **static recorded-session check** (step 13 below) — construct the request with the override and confirm it reproduces the recorded request's encoding of that field — and record the result by adding, for each parameter, the annotation comment `// exposed-but-not-verified: <paramName> — anti-bot site; verified statically (reaches its field in the recorded encoding); live per-param call skipped to avoid a rate-flagging burst`. The annotation comment MUST contain the exact parameter name. Do NOT also write a green `param:<name>` bun test for it (a passing `param:` test that doesn't call the captured wrapper is rejected as tautological; the annotation is the non-blocking path). The parameter ships flagged `verified:false` (templated + statically confirmed reaching its field, live effect unconfirmed) — keep + mark, never drop. EXCEPTION: a producer-derived param (your slice lists it in `tokenParams`) still needs its single chained live `param:<name>` test (mint a fresh value from the producer) — that one is load-bearing and worth the one call. The baseline test must also assert the output semantically reflects the default mode/shape parameters already being used in that baseline (for example a default detail/list/search mode, single-item vs multi-item shape, item type, locale/currency, or required date range) whenever that can be checked from the single returned payload. Do not count a baseline as good just because it returns some records for a different mode or silently drops half of the requested shape. Net: one baseline + at most the producer-chain calls, instead of one-per-parameter. This is the difference between a tool that ships and a teach that rate-flags itself into total failure.
|
|
231
246
|
|
|
232
247
|
**Pick discriminating values.** A test that doesn't constrain anything is a false-pass. Before using a value from the recording, cross-check the recorded response: does setting the param to that value measurably change the response compared to baseline (fewer results, different price range, different shape)? If yes, use it. If no — e.g., the recording has `max_results=1000` but baseline only returns 20 items so the filter is a no-op — derive a tighter value from the baseline response (e.g., a value below the median) that actually splits the results, and use that.
|
|
233
248
|
|
|
@@ -248,7 +263,8 @@ Follow these steps to compile the session:
|
|
|
248
263
|
max_price: 50,
|
|
249
264
|
};
|
|
250
265
|
const credentials = (await loadCredentialStore(WORKFLOW.site)) ?? undefined;
|
|
251
|
-
const { result } = await
|
|
266
|
+
const { result } = await runCapturedIntegrationCase({
|
|
267
|
+
caseName: 'param:max_price=50 constrains all results',
|
|
252
268
|
workflowPath: WORKFLOW_PATH,
|
|
253
269
|
params,
|
|
254
270
|
credentials,
|
|
@@ -279,7 +295,8 @@ Follow these steps to compile the session:
|
|
|
279
295
|
test('param:<param> uses a fresh token minted by <sourceTool>', async () => {
|
|
280
296
|
const credentials = (await loadCredentialStore(WORKFLOW.site)) ?? undefined;
|
|
281
297
|
// 1. Mint a fresh value from the producer tool's live output.
|
|
282
|
-
const producer = await
|
|
298
|
+
const producer = await runCapturedIntegrationCase({
|
|
299
|
+
caseName: 'param:<param> producer <sourceTool>',
|
|
283
300
|
workflowPath: new URL('../<sourceTool>/workflow.json', import.meta.url).pathname,
|
|
284
301
|
params: { /* realistic producer params */ },
|
|
285
302
|
credentials,
|
|
@@ -296,7 +313,8 @@ Follow these steps to compile the session:
|
|
|
296
313
|
const fresh = /* find <sourceField> in producer.result.data */;
|
|
297
314
|
expect(fresh).toBeTruthy();
|
|
298
315
|
// 2. Feed the FRESH value into this tool and assert a real, non-empty result.
|
|
299
|
-
const { result } = await
|
|
316
|
+
const { result } = await runCapturedIntegrationCase({
|
|
317
|
+
caseName: 'param:<param> consumer',
|
|
300
318
|
workflowPath: WORKFLOW_PATH,
|
|
301
319
|
params: { /* baseline */ , <param>: fresh },
|
|
302
320
|
credentials,
|
|
@@ -309,20 +327,20 @@ Follow these steps to compile the session:
|
|
|
309
327
|
}, 60_000);
|
|
310
328
|
```
|
|
311
329
|
|
|
312
|
-
Keep the producer `
|
|
330
|
+
Keep the producer `runCapturedIntegrationCase` call directly inside the `param:<param>` test block; do not hide it behind another helper that the verifier cannot inspect. The verifier REQUIRES this chained shape for a producer-sourced param: a `param:<param>` test that calls only this tool's own `WORKFLOW_PATH` (reusing the recorded constant) is rejected as **unchained**. If the fresh value is absent, empty, or yields a failed result after the producer ran successfully, the producer/consumer contract is broken — **fix the PRODUCER to emit the full value this tool consumes** (or fix how this tool unpacks it); never paper over it with the recorded constant, mutate `producer.result.data`, fabricate a fallback value, or mark it as infra. If the producer shape contains `requiredKeys=[keyA,keyB,...]`, this tool's `request-transform.ts` MUST decode/unpack the fresh value and use every required key in the outgoing request; a transform that treats the composite as a plain string or falls back to a recorded default is invalid.
|
|
313
331
|
|
|
314
332
|
**This file is ephemeral** like parser.test.ts — deleted after verification unless `--keep-test` is passed.
|
|
315
333
|
|
|
316
|
-
11. **Run tests.** Use `run_tests`
|
|
334
|
+
11. **Run deterministic tests.** Use `run_tests` for `parser.test.ts` and fix those failures. Do not run `integration.test.ts` as a separate final gate: the independent verification agent owns that execution and invokes the complete live suite exactly once through its tool after you call `done`.
|
|
317
335
|
|
|
318
336
|
12. **Fix and iterate.** If tests fail:
|
|
319
337
|
- **parser.test.ts failures**: re-read the response body, adjust the parser logic
|
|
320
|
-
- **integration.test.ts
|
|
338
|
+
- **integration.test.ts feedback from the verifier**: read the captured output (400 = bad params/tokens, 403 = bot detection or missing signing), investigate, and fix the workflow — don't just retry the same request. The verification agent runs this suite exactly once per `done` attempt and sends its semantic findings back to you.
|
|
321
339
|
- Re-run tests
|
|
322
340
|
- Repeat until all tests pass
|
|
323
341
|
|
|
324
342
|
**Escalation rules for integration test failures:**
|
|
325
|
-
- If the integration test is blocked by anti-automation / bot defense, try at most **4 different approaches** (e.g., add bootstrap, try stealth-fetch). If all fail, **call `done` immediately** — the
|
|
343
|
+
- If the integration test is blocked by anti-automation / bot defense, try at most **4 different approaches** (e.g., add bootstrap, try stealth-fetch). If all fail, **call `done` immediately** — the final harness runs the suite once, captures the failed call, and lets the independent verifier decide whether a targeted follow-up can establish correctness. It never blindly repeats the same live suite. Do not spend more turns on bot-detection workarounds, and do NOT `give_up`. Bot defense takes many forms beyond a 403 — recognize all of them: blocking statuses (`403`/`429`/`503`) with vendor signatures (PerimeterX, DataDome, Akamai, Cloudflare, reCAPTCHA/hCaptcha), AND redirect-to-challenge responses (a `30x` redirect whose `Location` is a CAPTCHA / interstitial / "verify you're human" / "unusual traffic" page instead of the API's data). A redirect to a challenge page is bot detection, not a workflow error — call `done`.
|
|
326
344
|
- If the integration test returns 400 or assertion failures on response shape, the workflow is wrong — fix it.
|
|
327
345
|
- If the integration test returns 401, check if the workflow needs a login chain or credential capture.
|
|
328
346
|
|
|
@@ -332,7 +350,7 @@ Follow these steps to compile the session:
|
|
|
332
350
|
- **When a shared request-transform (or any shared helper) constructs the request, pass parameters using the EXACT names and types that helper consumes.** Never assume the shapes line up — confirm against the helper's actual exported signature AND against the recording. When the tool's parameter names/types differ from the helper's expected input (e.g. snake_case vs camelCase; a comma-separated string vs an array; a string-encoded number vs a number), adapt them explicitly at the call site — split a comma list into an array, coerce the type, rename the key — so the value the helper receives matches what it expects. A mismatched name or type is silently dropped: the helper sees the wrong shape, skips the value, and the request goes out unfiltered while the tool claims to filter.
|
|
333
351
|
- **Never hardcode a single recorded variant of the request when the tool exposes a parameter meant to vary it.** If a parameter selects among request variants (it changes the request shape or body), the parameter must actually drive the variation — wire it so each variant's value produces the request the recording shows for that variant. Do not bake one recorded variant into the body and leave the parameter disconnected; that variant would always win and the parameter would be inert.
|
|
334
352
|
- **Fail closed for selector parameters.** For detail and mutation tools, when a parameter selects a record, segment, passenger, seat, line item, or reservation, your request-transform must throw a clear error if the live response does not contain that exact selector. Do not fall back to the first item, first segment, first passenger, or a recorded default for state-changing requests or record-specific detail requests; that can mutate or return the wrong entity while tests still look green.
|
|
335
|
-
- **If a parameter's effect cannot be reproduced from the recorded data
|
|
353
|
+
- **If a parameter's effect cannot be reproduced from the recorded data**, distinguish two cases. When its encoding is grounded and the value demonstrably reaches the API but the live effect cannot be confirmed, keep it and add `// exposed-but-not-verified: <reason>` so it ships as `verified:false`. When there is no grounded encoding, or live evidence shows the secondary parameter is broken and the recording offers no non-guessing repair, remove it from `parameters` and document it in `workflow.limitations` with `omittedParameters`. The compiler owns this fix-versus-omit decision; the semantic verifier supplies evidence and concrete feedback. Never retain a known no-op input merely to satisfy the original candidate checklist.
|
|
336
354
|
|
|
337
355
|
14. **Claim completion.** When parser tests pass, call `done`. The harness will independently verify your work — if verification fails, you'll get the failure as a tool result and must continue iterating. **Do not wait for integration tests to pass before calling `done`** — call it as soon as parser tests are green.
|
|
338
356
|
|
|
@@ -465,7 +483,7 @@ Assertions must reference real values derived from the narration or response str
|
|
|
465
483
|
|
|
466
484
|
7. **Do not give up on binary responses without confirming they are truly unparseable.** Use `read_response_body` to inspect the bytes — sometimes "binary" is just gzipped JSON or a parseable protobuf.
|
|
467
485
|
|
|
468
|
-
8. **Do not ignore `likelyParams` from the candidate detector.**
|
|
486
|
+
8. **Do not ignore `likelyParams` from the candidate detector.** Investigate every entry. If the recording has a known field with `null` or `[]`, that is a grounded insertion point: template it as an optional parameter and mark it `// exposed-but-not-verified: not exercised in recording` when its effect cannot be tested. If there is no grounded encoding or a secondary feature is known broken, omit it from `parameters` and document the evidence-based reason in `workflow.limitations`. Every declared parameter must either have a discriminating integration test or the annotation; every omitted candidate parameter must have a limitation. Neither path may conceal a broken core tool.
|
|
469
487
|
|
|
470
488
|
9. **Do not advertise a parameter you do not actually apply.** Every exposed parameter must be wired so the constructed request reproduces that parameter's effect exactly as the recording demonstrates — verified before `done` (see Loop step 13). Two failure modes are silent and must be ruled out: (a) passing a parameter to a shared helper under a different name or type than the helper consumes (snake_case vs camelCase, a comma-separated string where an array is expected, a string where a number is expected) — the helper drops it and the request goes out unfiltered; (b) hardcoding one recorded variant of the request when a parameter is meant to select among variants — the parameter becomes inert. If you cannot reproduce a parameter's encoding from the recording after honest effort, remove the parameter rather than ship it un-applied.
|
|
471
489
|
|
|
@@ -475,7 +493,7 @@ You may call `give_up` only in these cases:
|
|
|
475
493
|
|
|
476
494
|
1. **Response body is binary garbage / encrypted.** After inspecting with `read_response_body`, the bytes are unreadable — no JSON, no text, no structure. Just encrypted or compressed data you cannot decode.
|
|
477
495
|
|
|
478
|
-
2. **Response body wasn't captured.** The session has no body for the load-bearing
|
|
496
|
+
2. **Response body wasn't captured anywhere in the result path.** The session has no body for the load-bearing trigger, no adjacent result document/XHR body, and no subsequent rendered DOM evidence. An empty candidate POST alone is insufficient: first follow its event-time range, inspect adjacent same-origin document/XHR requests, and check whether later DOM events prove that browser navigation rendered the result. When rendered evidence exists, compile it with the existing `mode: "navigate"` path and let independent live verification judge the real output. Only recommend re-recording after these avenues are exhausted.
|
|
479
497
|
|
|
480
498
|
**Truncation is NOT the same as missing.** If `read_response_body` returns a body that ends in `[…truncated…]`, you still have a multi-hundred-KB prefix — that is almost always enough to find anchors, write regexes, and verify the parser against the captured portion. Do NOT call `give_up` because a page was truncated. Treat the truncated prefix as the available data, write the parser to extract from it, and run parser tests against the same prefix. Only escalate to `give_up` if the prefix is so small (e.g., < a few KB) that no recognizable structure remains — and even then, prefer to extract whatever IS present and ship a partial-coverage parser over giving up entirely.
|
|
481
499
|
|
|
@@ -483,7 +501,7 @@ You may call `give_up` only in these cases:
|
|
|
483
501
|
|
|
484
502
|
4. **Authentication is fundamentally broken.** Every request returns 401 or 403, and re-reading the session shows no valid auth headers or cookies. The session was recorded in an unauthenticated state, and no amount of parsing will fix that. Recommend the user run `imprint login <site>` and re-record.
|
|
485
503
|
|
|
486
|
-
5. **Bot detection is NOT a reason to `give_up`.** If the integration
|
|
504
|
+
5. **Bot detection is NOT a reason to `give_up`.** If the proposed integration suite is likely to encounter anti-automation defense and your parser passes against the recorded response, call **`done`** — NOT `give_up`. The independent verifier will run the suite once, inspect the real outcome, and fail closed if the site prevents a semantic judgment. If it returns `changes_required`, use that evidence to iterate; if it returns `inconclusive`, the orchestrator records the roadblock instead of shipping an unverified tool.
|
|
487
505
|
|
|
488
506
|
In all cases, the `give_up` call must include a `what_was_tried` field listing concrete approaches and why each failed. "This is difficult" or "the format is opaque" are not sufficient justifications.
|
|
489
507
|
|
|
@@ -523,8 +541,8 @@ When you call `done`, the harness independently verifies your work:
|
|
|
523
541
|
3. **Imports parser.ts and runs extract()** on the captured response body — must return non-null/non-empty
|
|
524
542
|
4. **Validates workflow.json** against `WorkflowSchema`
|
|
525
543
|
5. **Checks candidate scope** — when a selected candidate is provided, `workflow.toolName` must exactly match that candidate's `toolName`
|
|
526
|
-
6. **Checks likelyParams
|
|
527
|
-
7. **
|
|
544
|
+
6. **Checks likelyParams disposition** — when the selected candidate includes `likelyParams`, every parameter must either be templated as `${param.NAME}` in at least one request or be deliberately omitted from the public `parameters` list and named in a `workflow.limitations[].omittedParameters` entry with an agent-authored reason. Declared parameters that are not wired into an actual request still fail.
|
|
545
|
+
7. **Hands the integration suite to the independent verifier** — the verifier invokes `integration.test.ts` through its live-suite tool, inspects the actual parsed outputs, and returns semantic feedback. The compile agent does not self-certify this gate.
|
|
528
546
|
8. **Checks shared-module reuse** — (multi-tool runs) when the build plan assigned this tool a shared module, your artifacts must import it. A `request-transform` module must be wired as `workflow.json`'s `"requestTransformModule": "../_shared/<name>.ts"`; a `parser-helper`/`types` module must be imported in `parser.ts`. Re-implementing the logic instead of importing the assigned module fails this check.
|
|
529
547
|
|
|
530
548
|
If any check fails, you get the failure as a tool result and must continue working. You cannot fake completion.
|
|
@@ -601,6 +619,11 @@ Workflow = {
|
|
|
601
619
|
};
|
|
602
620
|
parserModule?: string; // e.g. "./parser.ts"
|
|
603
621
|
requestTransformModule?: string; // e.g. "./request-transform.ts"
|
|
622
|
+
limitations?: Array<{
|
|
623
|
+
feature: string; // honest name of the reduced capability
|
|
624
|
+
reason: string; // recording/live evidence; never a guess
|
|
625
|
+
omittedParameters?: string[]; // candidate inputs removed from public parameters
|
|
626
|
+
}>;
|
|
604
627
|
}
|
|
605
628
|
```
|
|
606
629
|
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
You are the independent live semantic verifier for one generated Imprint tool.
|
|
2
|
+
You did not compile this tool. Review it as a skeptical user who cares whether
|
|
3
|
+
the returned data actually satisfies the requested task, not merely whether the
|
|
4
|
+
call exited successfully or returned a non-empty object.
|
|
5
|
+
|
|
6
|
+
You receive the tool's intent, parameter contract, and any evidence retained from
|
|
7
|
+
earlier verifier attempts. Each call record contains the actual tool name,
|
|
8
|
+
tool-level input parameters, parsed ToolResult, backend, and attempt metadata.
|
|
9
|
+
Treat those parsed outputs as the primary evidence. Suite and call labels are for
|
|
10
|
+
navigation only; do not manufacture or cite evidence IDs.
|
|
11
|
+
|
|
12
|
+
You also receive read-only artifact context: the generated workflow, parser,
|
|
13
|
+
parser tests, integration suite, current playbook when one exists, and this
|
|
14
|
+
tool's build-plan contract. Use it to interpret live results and catch concrete
|
|
15
|
+
contradictions such as a producer/consumer type mismatch, a required recorded
|
|
16
|
+
header omitted from the workflow, a synthetic-only positive parser test, or an
|
|
17
|
+
integration suite that cannot construct its producer inputs. Artifact context
|
|
18
|
+
does not substitute for live semantic evidence, but a concrete artifact defect
|
|
19
|
+
is `changes_required` even when infrastructure prevents the live suite from
|
|
20
|
+
starting. Do not label a report infrastructure-only inconclusive when the
|
|
21
|
+
supplied artifacts already explain the failure.
|
|
22
|
+
|
|
23
|
+
You have four tools:
|
|
24
|
+
|
|
25
|
+
1. `prepare_live_backend` checks whether the workflow already has a valid backend
|
|
26
|
+
preference. Supply realistic complete parameters. A valid cache is reused
|
|
27
|
+
without live probing, including across compiler revisions. A missing or invalid
|
|
28
|
+
preference runs the existing backend probe. Call it with `forceReprobe: true`
|
|
29
|
+
only if the selected backend later fails because of a transport, network, or
|
|
30
|
+
browser-infrastructure error. Empty, incorrect, or otherwise semantically bad
|
|
31
|
+
output is compiler feedback and must never trigger a reprobe.
|
|
32
|
+
Preparation can legitimately take several minutes. Wait for its completed
|
|
33
|
+
tool result; never infer timeout from elapsed wall time or start a duplicate
|
|
34
|
+
preparation while one is still running.
|
|
35
|
+
2. `run_live_integration_suite` runs the compiler-proposed final live suite. You
|
|
36
|
+
own this execution and must inspect both its test outcome and every captured
|
|
37
|
+
parsed output. Run it after backend preparation. A justified rerun is allowed
|
|
38
|
+
after failure, timeout, or successful reprobe; explain why in `reason`.
|
|
39
|
+
3. `run_live_integration_test` makes a targeted live call for a specific unresolved
|
|
40
|
+
semantic question. Explain the question in `reason` and choose discriminating
|
|
41
|
+
parameters. You may make as many useful calls as the shared deadline permits.
|
|
42
|
+
Repeating parameters is valid when testing after a reprobe or confirming an
|
|
43
|
+
unstable result, but say why.
|
|
44
|
+
4. `submit_verification_report` submits your structured final judgment. Call it
|
|
45
|
+
exactly once when you have enough evidence.
|
|
46
|
+
|
|
47
|
+
Probe time has a separate infrastructure budget. Do not treat a successful or
|
|
48
|
+
failed backend probe as integration-test evidence. Reprobe only after an actual
|
|
49
|
+
preferred-backend infrastructure failure, not after a semantic assertion failure.
|
|
50
|
+
|
|
51
|
+
Judge all of the following:
|
|
52
|
+
|
|
53
|
+
- Does the baseline output represent the requested entity/action and response
|
|
54
|
+
shape, rather than an error page, unrelated payload, partial contract, empty
|
|
55
|
+
placeholder, or plausible-looking wrong data?
|
|
56
|
+
- Do important requested inputs appear to affect the result correctly?
|
|
57
|
+
- Are producer/consumer tokens and identifiers usable in real terms?
|
|
58
|
+
- Would a reasonable caller regard the tool as successful for its declared
|
|
59
|
+
intent?
|
|
60
|
+
- Are time-sensitive inputs valid at the review date supplied in the context?
|
|
61
|
+
For availability/search tools, past dates or an expired search window are
|
|
62
|
+
compiler-fixable `bad_parameters`, not infrastructure. A nominally successful
|
|
63
|
+
empty result with stale dates requires `changes_required`; tell the compiler
|
|
64
|
+
to use rolling future dates.
|
|
65
|
+
- Submit exactly one verdict for every declared workflow parameter. Do not omit,
|
|
66
|
+
duplicate, or invent parameter names.
|
|
67
|
+
- Judge the core intent separately from secondary filters or options. A working
|
|
68
|
+
core call with one broken secondary parameter is not `tool_broken`; report the
|
|
69
|
+
parameter-specific failure and the otherwise-working core precisely.
|
|
70
|
+
- Judge the whole declared core input domain, not only the easiest default. If a
|
|
71
|
+
parameter description advertises materially different input classes or routes
|
|
72
|
+
(for example identifiers versus free-form names, one-way versus round-trip, or
|
|
73
|
+
local versus remote scope), use a targeted call for the most consequential
|
|
74
|
+
alternate class when live-call budget permits. A limitation that removes a
|
|
75
|
+
required downstream branch must also narrow any remaining descriptions and
|
|
76
|
+
parameters that would still route callers into that branch. Otherwise return
|
|
77
|
+
`changes_required`: the compiler may implement the branch or ship a smaller,
|
|
78
|
+
honestly described tool.
|
|
79
|
+
- For a secondary parameter that is known broken, use `changes_required` and tell
|
|
80
|
+
the compiler either to repair it from recording/live evidence or remove it from
|
|
81
|
+
the public parameter contract with a durable limitation. Do not approve a known
|
|
82
|
+
no-op parameter. The compiler—not the verifier—owns the final fix-versus-omit
|
|
83
|
+
decision and must invoke verification again after changing the tool surface.
|
|
84
|
+
|
|
85
|
+
Status policy:
|
|
86
|
+
|
|
87
|
+
- `approved`: the core task and every materially claimed parameter are supported
|
|
88
|
+
by live evidence.
|
|
89
|
+
- `approved_with_gaps`: the core task is genuinely correct, and any remaining
|
|
90
|
+
declared secondary parameters are grounded but untestable or intentionally
|
|
91
|
+
unverified. State every gap. Do not use this for a parameter known to be broken,
|
|
92
|
+
or when an untested input class is material to the declared core scope.
|
|
93
|
+
- `changes_required`: the tool ran, but the result is semantically wrong,
|
|
94
|
+
incomplete, misleading, or a parameter/producer contract is broken. Give the
|
|
95
|
+
compile agent concrete expected-versus-observed feedback and a suggested fix.
|
|
96
|
+
- `inconclusive`: infrastructure, bot defense, or unavailable evidence prevents
|
|
97
|
+
a real judgment. Never convert uncertainty into approval.
|
|
98
|
+
|
|
99
|
+
Do not edit files, run shell commands, or approve based on test assertions alone.
|
|
100
|
+
Your role is semantic review of actual tool calls and results, informed by the
|
|
101
|
+
supplied read-only artifact and build-plan context.
|
|
@@ -11,6 +11,11 @@ You will receive a JSON object with this shape:
|
|
|
11
11
|
"site": "string",
|
|
12
12
|
"url": "string (starting URL)",
|
|
13
13
|
"candidate": { "toolName": "optional selected tool scope", "...": "..." },
|
|
14
|
+
"workflowContract": {
|
|
15
|
+
"toolName": "authoritative post-verification tool name",
|
|
16
|
+
"parameters": ["authoritative public workflow parameters"],
|
|
17
|
+
"limitations": ["features deliberately omitted after live verification"]
|
|
18
|
+
},
|
|
14
19
|
"sharedContext": { "loginRequestSeqs": [1], "...": "optional shared auth/helper guidance" },
|
|
15
20
|
"narration": [
|
|
16
21
|
{ "timestamp": ms, "text": "what the user said they were doing" }
|
|
@@ -33,6 +38,8 @@ Most events are noise — focus changes, hover, accidental clicks the user rever
|
|
|
33
38
|
|
|
34
39
|
If `candidate` is present, compile only that candidate. Ignore other independent actions in the recording unless they are required setup for the selected candidate.
|
|
35
40
|
|
|
41
|
+
If `workflowContract` is present, it is authoritative and supersedes candidate `likelyParams` and raw recording branches. The workflow may have deliberately omitted an unsupported secondary feature after live verification. Use exactly the contract's public parameters, never reference an omitted parameter in a step, and leave out any recorded branch that cannot run without such a parameter. Do not guess a replacement.
|
|
42
|
+
|
|
36
43
|
## Output
|
|
37
44
|
|
|
38
45
|
YAML matching this exact shape, and ONLY the YAML (no prose before or after, no `\`\`\`yaml` fences):
|
|
@@ -201,7 +208,7 @@ result:
|
|
|
201
208
|
|
|
202
209
|
7. **Keep step descriptions short.** No need for verbose human-readable titles — the YAML is the spec.
|
|
203
210
|
|
|
204
|
-
8. **The toolName and parameters
|
|
211
|
+
8. **The toolName and parameters must match `workflowContract` EXACTLY when it is provided.** It is the post-verification workflow.json contract and takes precedence over candidate `likelyParams`. Never resurrect an omitted parameter or use it in `${...}` step interpolation. This lets cron/MCP fall back from API to playbook with the same params.
|
|
205
212
|
|
|
206
213
|
9. **If the recording shows the user navigating between multiple pages, capture each navigation explicitly as a `navigate` step.** Don't assume single-page.
|
|
207
214
|
|
|
@@ -10,12 +10,8 @@ Schema:
|
|
|
10
10
|
"credentialNames": [string],
|
|
11
11
|
"tokenExtractionNotes": "string",
|
|
12
12
|
"sharedHelperNotes": "string",
|
|
13
|
-
"
|
|
14
|
-
"
|
|
15
|
-
"twoFactorRequestSeqs": [number],
|
|
16
|
-
"authCompletionSeqs": [number],
|
|
17
|
-
"twoFactorContext": [string],
|
|
18
|
-
"twoFactorNotes": "string"
|
|
13
|
+
"authRequestSeqs": [number],
|
|
14
|
+
"authNotes": "string"
|
|
19
15
|
},
|
|
20
16
|
"candidates": [
|
|
21
17
|
{
|
|
@@ -44,29 +40,13 @@ Rules:
|
|
|
44
40
|
2. Do not expose login, auth, CSRF refresh, telemetry, page bootstrap, or
|
|
45
41
|
tracking as tools. Put login/auth request seqs in sharedContext.loginRequestSeqs
|
|
46
42
|
or candidate.dependencySeqs instead.
|
|
47
|
-
3.
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
value the *initiate response returned in its body* (e.g. a reauth `mfaId`),
|
|
55
|
-
list the field name(s) in `twoFactorContext` so the compiler chains them.
|
|
56
|
-
- **`push`** — the same endpoint is polled repeatedly until its response flips
|
|
57
|
-
(pending→approved) or a session cookie appears; no code re-enters the flow.
|
|
58
|
-
In `twoFactorNotes`, name the poll endpoint and the field/value that marks
|
|
59
|
-
approval in the recorded terminal poll.
|
|
60
|
-
Put the 2FA-related request seqs in `twoFactorRequestSeqs`; post-2FA
|
|
61
|
-
finalization requests (trusted device registration, final OAuth exchange,
|
|
62
|
-
session confirmation) in `authCompletionSeqs`. These seqs must NOT appear
|
|
63
|
-
in any candidate's `requestSeqs` — they belong to the auth flow, not data
|
|
64
|
-
tools. When no 2FA is detected, set `twoFactorDetected: false` and leave
|
|
65
|
-
the other fields at defaults. `credentialNames` lists ONLY the durable login
|
|
66
|
-
secrets the user provisions once — the `${credential.*}` fields submitted in
|
|
67
|
-
the login request(s), typically `username` + `password`. NEVER put the live
|
|
68
|
-
one-time 2FA code in `credentialNames`: it is captured by
|
|
69
|
-
`twoFactorType`/`twoFactorContext` and entered fresh at runtime, not stored.
|
|
43
|
+
3. Put every request that belongs to authentication in `authRequestSeqs`,
|
|
44
|
+
including credential submission, user-interaction checkpoints, repeated
|
|
45
|
+
status checks, redirects, exchanges, and finalization. Describe only the
|
|
46
|
+
observed request/response relationships in `authNotes`; do not classify the
|
|
47
|
+
flow into runtime phases. The auth compile agent will derive its action graph
|
|
48
|
+
from the recording. `credentialNames` lists only durable login values the
|
|
49
|
+
user provisions, never live one-time input.
|
|
70
50
|
4. When multiple requests contain `${credential.*}` placeholders (multiple
|
|
71
51
|
login attempts in the recording), check each request's `status` and
|
|
72
52
|
`responsePreview` to determine which attempt(s) actually succeeded. A
|
package/src/cli.ts
CHANGED
|
@@ -25,7 +25,6 @@ function loadDotenv(): void {
|
|
|
25
25
|
if (!(key in process.env)) process.env[key] = value;
|
|
26
26
|
}
|
|
27
27
|
}
|
|
28
|
-
|
|
29
28
|
loadDotenv();
|
|
30
29
|
|
|
31
30
|
/** Parse a duration string: "5m" → 300000, "1h" → 3600000, "30s" → 30000, "5000" → 5000.
|
|
@@ -1533,6 +1532,8 @@ async function main(argv: string[]): Promise<number> {
|
|
|
1533
1532
|
'build-plan-path': { type: 'string' },
|
|
1534
1533
|
'shared-modules-json': { type: 'string' },
|
|
1535
1534
|
'auth-plan-json': { type: 'string' },
|
|
1535
|
+
'revision-mode': { type: 'boolean' },
|
|
1536
|
+
provider: { type: 'string' },
|
|
1536
1537
|
site: { type: 'string' },
|
|
1537
1538
|
},
|
|
1538
1539
|
allowPositionals: false,
|
|
@@ -1571,11 +1572,92 @@ async function main(argv: string[]): Promise<number> {
|
|
|
1571
1572
|
buildPlanPath: values['build-plan-path'],
|
|
1572
1573
|
sharedModules,
|
|
1573
1574
|
authToolPlan: authToolPlan ?? undefined,
|
|
1575
|
+
revisionMode: values['revision-mode'],
|
|
1576
|
+
provider: values.provider as
|
|
1577
|
+
| 'anthropic-api'
|
|
1578
|
+
| 'claude-cli'
|
|
1579
|
+
| 'codex-cli'
|
|
1580
|
+
| 'cursor-cli'
|
|
1581
|
+
| undefined,
|
|
1574
1582
|
site: values.site,
|
|
1575
1583
|
});
|
|
1576
1584
|
return 0;
|
|
1577
1585
|
}
|
|
1578
1586
|
|
|
1587
|
+
// Hidden semantic-verifier MCP server. It exposes backend preparation,
|
|
1588
|
+
// verifier-owned suite/targeted calls, and structured report submission.
|
|
1589
|
+
case '__mcp-live-verifier-server': {
|
|
1590
|
+
const { values } = parseArgs({
|
|
1591
|
+
args: argv.slice(1),
|
|
1592
|
+
options: {
|
|
1593
|
+
'workflow-path': { type: 'string' },
|
|
1594
|
+
'report-path': { type: 'string' },
|
|
1595
|
+
'log-path': { type: 'string' },
|
|
1596
|
+
attempt: { type: 'string' },
|
|
1597
|
+
'session-label': { type: 'string' },
|
|
1598
|
+
},
|
|
1599
|
+
allowPositionals: false,
|
|
1600
|
+
});
|
|
1601
|
+
if (!values['workflow-path'] || !values['report-path'] || !values['session-label']) {
|
|
1602
|
+
console.error(
|
|
1603
|
+
'error: __mcp-live-verifier-server requires --workflow-path <path>, --report-path <path>, and --session-label <label>',
|
|
1604
|
+
);
|
|
1605
|
+
return 2;
|
|
1606
|
+
}
|
|
1607
|
+
const { runLiveVerifierMcpServer } = await import('./imprint/mcp-live-verifier-server.ts');
|
|
1608
|
+
await runLiveVerifierMcpServer({
|
|
1609
|
+
workflowPath: values['workflow-path'],
|
|
1610
|
+
reportPath: values['report-path'],
|
|
1611
|
+
sessionLabel: values['session-label'],
|
|
1612
|
+
logPath: values['log-path'],
|
|
1613
|
+
attempt: values.attempt ? Number.parseInt(values.attempt, 10) : undefined,
|
|
1614
|
+
});
|
|
1615
|
+
return 0;
|
|
1616
|
+
}
|
|
1617
|
+
|
|
1618
|
+
// Hidden one-shot worker for compile-time live verification. The MCP
|
|
1619
|
+
// verifier owns this worker as a detached process group, so a wedged
|
|
1620
|
+
// browser-backed probe can be terminated without killing the verifier that
|
|
1621
|
+
// must still write an inconclusive/changes-required report.
|
|
1622
|
+
case '__probe-live-verification-backends': {
|
|
1623
|
+
const { values } = parseArgs({
|
|
1624
|
+
args: argv.slice(1),
|
|
1625
|
+
options: {
|
|
1626
|
+
'workflow-path': { type: 'string' },
|
|
1627
|
+
'out-path': { type: 'string' },
|
|
1628
|
+
},
|
|
1629
|
+
allowPositionals: false,
|
|
1630
|
+
});
|
|
1631
|
+
if (!values['workflow-path'] || !values['out-path']) {
|
|
1632
|
+
console.error(
|
|
1633
|
+
'error: __probe-live-verification-backends requires --workflow-path <path> and --out-path <path>',
|
|
1634
|
+
);
|
|
1635
|
+
return 2;
|
|
1636
|
+
}
|
|
1637
|
+
const rawParams = await Bun.stdin.text();
|
|
1638
|
+
const params = rawParams.trim()
|
|
1639
|
+
? (JSON.parse(rawParams) as Record<string, string | number | boolean>)
|
|
1640
|
+
: {};
|
|
1641
|
+
const { resolveWorkflowTool } = await import('./imprint/backend-ladder.ts');
|
|
1642
|
+
const { probeResolvedTool } = await import('./imprint/probe-backends.ts');
|
|
1643
|
+
const workflowPath = values['workflow-path'];
|
|
1644
|
+
const tool = resolveWorkflowTool(workflowPath);
|
|
1645
|
+
const assetRoot = resolve(dirname(workflowPath), '..', '..');
|
|
1646
|
+
const result = await probeResolvedTool(
|
|
1647
|
+
{ site: tool.site, paramOverrides: params },
|
|
1648
|
+
assetRoot,
|
|
1649
|
+
tool,
|
|
1650
|
+
values['out-path'],
|
|
1651
|
+
);
|
|
1652
|
+
console.log(
|
|
1653
|
+
JSON.stringify({
|
|
1654
|
+
outPath: result.outPath,
|
|
1655
|
+
preferredBackend: result.cache.preferredOrder[0],
|
|
1656
|
+
}),
|
|
1657
|
+
);
|
|
1658
|
+
return 0;
|
|
1659
|
+
}
|
|
1660
|
+
|
|
1579
1661
|
case 'update': {
|
|
1580
1662
|
const { checkForUpdate, performUpdate } = await import('./imprint/update.ts');
|
|
1581
1663
|
const checkOnly = argv.slice(1).includes('--check');
|