@reclaimprotocol/agent 0.1.23 → 0.1.25

package/skill/builder-mode.md

@@ -1,17 +1,60 @@
-This server targets the Reclaim BUILDER backend. It exposes three tool layers: generated API tools (one per builder operation, snake_case), credential/verification helpers, and browser agent tools (CDP capture, synthesis, replay, proof).
-
-**Authentication:** try `get_me` first. On 401, `authenticate({})` opens a device-pairing session — give the user the `attachUrl`, then poll `authenticate({ code })` until LINKED. Tokens are cached ~2 days.
-
-**Provider-authoring flow:** `attach_browser` (local, Builder remote, or custom CDP) → `start_capture(tabId)` ⇒ a **`captureId`** (the session handle every browser/inspect tool takes as its first arg) → drive the page (`navigate`, `wait_for_page`, `eval_in_page`, `get_cookies`) → inspect (`list_requests`, `get_request`, `find_requests_containing`) → `propose_provider` ⇒ a **`draftId`** (the recipe + captured secrets held server-side; secrets never enter the transcript) → `replay_request(captureId, draftId)` → `run_proof(captureId, draftId, ownerAddress)` → resolve a `providerId` (**Reuse before create**, below) → `create_provider_version_from_capture` (lands on the draft branch) → `publish_provider_version` (PRIVATE) or `submit_provider_version_for_review` (PUBLIC).
-
-**Reuse before create (avoid duplicates):** `create_provider_version_from_capture` requires a `providerId`. Before minting a new one with `create_provider`, search your existing providers with `providers` (by domain / data point) — a site you've already built will be there. If one matches, pass its `providerId` so the capture lands as a NEW VERSION on that provider. Only `create_provider` (then use its returned id) when none exists. Re-running the flow against `create_provider` each time is what stacks duplicate providers in "My providers".
-
-**Multiple requests & login-gated data:** one version can carry many requests — `propose_provider` + `run_proof` each target request individually, then publish them together via `create_provider_version_from_capture({ providers: [draftA, draftB, …] })` (every entry is required at verification; the single-`provider` form still works for the one-request case). For data gated behind login/navigation, supply `jsUserScripts` — plain page JavaScript if needed that drives login/waits for user to login/waits for user to do multifactor auth/navigation with standard browser APIs (`window.location`, clicking elements, submitting forms) so the target request fires (page requests are intercepted automatically; `window.Reclaim` is only for rare edge cases) — and/or `allowedJsRequests` (extra requests the script may fire, e.g. pagination). NOTE: the builder verification client installs `jsUserScripts` to run before every page load (before navigating to `initialUrl`), matching the old-devtool `customInjection`; `window.Reclaim` itself is not yet exposed there, so prefer plain browser JS.
-
-**Templating:** bare `{{name}}` placeholders are provider params (values extracted from the response by the attestor). Values the CONSUMER must supply at verification time (e.g. a username in the URL path) are templated `{{context.<name>}}` and declared on the version as `requiredContext` — a JSON Schema set via `update_provider_version` while the version is on draft. The Builder validates every session-create's `context` against it (400 on mismatch; extra context fields are always allowed). `requiredContext` is optional — skip it when the recipe has no `{{context.*}}` placeholders. At verification time, requests are matched by exact origin+path: `{{context.<name>}}` segments match the consumer-sent value only, bare `{{name}}` values are extracted from the live traffic (plain-text alignment — works for JSON, XML, HTML, form-encoded, any text). Extract selectors apply client-side in spec order xPath → jsonPath → regex, each to the prior selection, so only the first selector constrains the full response; responses that cannot satisfy it (or miss a required match) are skipped. `propose_provider` returns advisory `verificationWarnings` for author-only decisions — surface them to the dev before publishing. **Two hard limitations:** (1) never `hash`/OPRF a target whose value also appears in the request URL — URL param values are always signed into context in the clear, so `propose_provider` throws rather than shipping a draft that can't verify; (2) ANY param — bare `{{SECRET_x}}` or consumer-supplied `{{context.SECRET_apiKey}}` alike — named with `SECRET` routes to `secretParams.paramValues` and CAN be used in the URL/geoLocation, but only up to a small TOTAL character budget summed across every secret value in that URL — 2 chars under `key-update` (default), 12 under `zk` — checked at verification time against the real value, since it doesn't exist yet at publish time. See `how_it_works` for the full reasoning.
-
-**Discover & resolve (find providers for the user's app):** when a dev wants to add verification but doesn't know which providers to use, run the same flow the Discover UI does. Ask which website their users prove data from, then `discover_data_points({ domain })` to list what's verifiable there. Confirm the data points with the dev, then `discover_resolve({ domain, dataPoints })` → the smallest provider set (`providers`, greedy combine), the `requiredContext` inputs to collect from the dev, and any `uncovered` points (author those with the provider-authoring flow above, then re-resolve). Create the session with `create_verification_session({ providers: [{ providerId }, …], context, verificationClientUrl })`, passing EVERY resolved provider id and the collected context. `merge.mergeable` only signals the set could collapse into one provider — you still pass them all; the Builder runs them in one session.
-
-**Verification sessions:** consumers authenticate with a per-org token (`rorg_…`, issued in the dashboard via `POST /orgs/{orgId}/token`) and create sessions via `POST /verifications/sessions`. Results are ECIES-encrypted (secp256k1 + AES-256-GCM) to the org's eth public key when its `canEncryptResult` is enabled — set via `PUT /orgs/{orgId}/keypair` with an eth public key — and delivered to the org's callback subscriptions (plaintext otherwise). Use the org/session tools (`get_verification_session`, `list_verification_events`, `list_org_verification_sessions`) to troubleshoot.
-
-**Backend mode:** this server can alternatively run against the devtools backend with a reduced tool set (no generated API tools, no orgs/review flows). `get_devtools_mode` reports the active mode and why it's in effect; `set_devtools_mode` persists a switch to `~/.reclaim/config.json` — it takes effect on the NEXT server start (restart/reconnect), and an explicit `USE_OLD_DEVTOOLS` env var overrides the persisted choice.
+Reclaim provider-authoring tools, running against the BUILDER backend. Three
+tool layers: generated API tools (one per builder operation, snake_case),
+credential/verification helpers, and browser agent tools (CDP capture,
+synthesis, replay, proof).
+
+**A `provider` is a recipe for extracting a data point from a site. A
+`verification` is an end-user session that proves a data point using one. Never
+call a provider a "verification."**
+
+**Read the guide before authoring.** `how_it_works({ topic })` serves it, one
+topic at a time — call `how_it_works({})` for the topic index. Topics:
+`overview`, `auth`, `browser`, `capture`, `draft`, `params`, `prove`, `publish`,
+`discover`, `user-script`, `interception`, `hash-validation`, `credentials`,
+`never-do`, `troubleshooting`.
+
+**Authentication:** try `get_me` first. On 401, `authenticate({})` returns a
+device-pairing `code` + `attachUrl` — give the user the URL, then poll
+`authenticate({ code })` until LINKED. Tokens cache ~2 days.
+(→ `how_it_works({ topic: "auth" })`)
+
+**Ask who is doing the signing in, then offer a browser.** "Yourself, or does
+someone else need to log in?" decides half of `attach_browser`'s list. Doing it
+themselves → `dedicated` or `attach`, both local, neither shareable. Someone
+else, or a phone-only site → `container` (the Reclaim runtime in Docker, free,
+shareable with `share_browser_view`) or `builder` (hosted, no install,
+**chargeable**). Say which costs money, and say `container` exists at all:
+nobody guesses it.
+
+**The flow:** `attach_browser` (ask the developer which browser source — local
+Chrome, the Reclaim runtime in Docker, a remote Builder browser, or a CDP URL
+they supply) → `start_capture`
+⇒ **`captureId`** (the handle every browser/inspect tool takes first) → drive
+the page (`navigate`, `wait_for_page`, `eval_in_page`, `get_cookies`) → inspect
+(`list_requests`, `get_request`, `find_requests_containing`) →
+`propose_provider` ⇒ **`draftId`** (recipe + captured secrets, held server-side
+— secrets never enter the transcript) → `replay_request(captureId, draftId)` →
+`run_proof(captureId, draftId, ownerAddress)` → resolve a `providerId` →
+`create_provider_version_from_capture` (lands on the draft branch) →
+`publish_provider_version` (PRIVATE) or `submit_provider_version_for_review`
+(PUBLIC).
+
+**Reuse before create.** `create_provider_version_from_capture` needs a
+`providerId`. Search your existing providers with `providers` (by domain / data
+point) BEFORE calling `create_provider` — routing every run through
+`create_provider` is what stacks duplicates in "My providers".
+
+**Never prove a public endpoint.** After `replay_request` succeeds, re-run it
+with `withoutSecrets: true`. If the value still extracts, the endpoint is public
+and the proof binds to nobody — find an auth-bound request instead.
+
+**Two hard constraints when drafting** (details in topics `draft` / `params`):
+never `hash`/OPRF a value that also appears in the request URL; any param named
+with `SECRET` may be used in a URL only within a tiny TOTAL character budget (2
+under `key-update`, 12 under `zk`), checked at verification time.
+
+**Backend mode:** this server can instead run against the devtools backend with
+a reduced tool set. `get_devtools_mode` reports the active mode and why;
+`set_devtools_mode` persists a switch to `~/.reclaim/config.json`, effective on
+the NEXT server start. An explicit `USE_OLD_DEVTOOLS` env var wins over the
+persisted choice.

package/skill/hash-validation.md

@@ -0,0 +1,170 @@
+# Spec hashing: `bodySniff` and `verifyProof({ providerId })`
+
+## Contents
+
+- The one fact everything follows from
+- `bodySniff` — when to enable it, how to template a partly dynamic body
+- How `verifyProof(proof, { providerId })` gates a proof
+- Why `url` is hashed verbatim, and when that makes
+  `allowedJsRequests`/`allowedInjectedRequestData` unusable
+- Optional fields, and the `dangerouslyDisableContentValidation` fallback
+- Related topics
+
+Everything here follows from one fact, confirmed directly from
+`hashProviderParams` in `@reclaimprotocol/attestor-core`:
+
+> The claim/spec hash is
+> `keccak256(canonicalStringify({ url, method, body, responseMatches, responseRedactions }))`,
+> and attestor-core does **no** template resolution, substitution, or
+> `{{}}`-stripping on any of it. It hashes exactly the strings it is given.
+
+## `bodySniff`
+
+A per-request field (it sits next to `url`/`method` on each `requestData` entry,
+not a version-level setting): `{ enabled: boolean, template: string }`.
+
+**`bodySniff.enabled` should be `true` whenever the request HAS a body at all**
+— it isn't reserved for dynamic bodies. Only leave it off when the request
+genuinely sends no body (typically a GET). Both publish tools in this package
+already set `enabled`/`template` from `provider.body` for a normal
+capture-and-replay provider, so you don't touch it there.
+
+The rest of this section matters when authoring a claim through live
+interception (`window.Reclaim.requestClaim`) whose body isn't the one literal
+string you captured once — usually because a SMALL part is dynamic per
+user/run.
+
+`bodySniff` as a concept doesn't exist inside attestor-core at all: it's
+translated one layer up (old-devtools' `providerUtils.ts`:
+`body: bodySniff.enabled ? bodySniff.template : ''`), and THAT literal string —
+including any unresolved `{{paramName}}` text still in it — is what reaches
+`hashProviderParams`.
+
+**The consequence, and why this differs from `templateParams`.** A live claim's
+`body` can also be set to the literal template string containing
+`{{paramName}}` rather than the substituted content, with the real value
+supplied separately as a witness param. Since the body is hashed opaquely, a
+claim whose `body` is template `T` and a spec whose `bodySniff.template` is that
+same `T` hash identically, whatever the placeholder stood for. This is the
+OPPOSITE of `templateParams`/`${var}` in `responseMatches`/`responseRedactions`,
+which — per the JS SDK's `generateSpecsFromRequestSpecTemplate` — genuinely IS
+substituted before hashing. Don't conflate them: `${var}` in a match/redaction
+is resolved before hashing; `{{var}}` in a body is never resolved, on either
+side.
+
+**Practical use.** Work out exactly which part of the body varies across
+users/runs — usually one small field, not the whole payload. Template ONLY that
+part; leave everything else literal. For
+`{"request":1,"type":"user_info","id":"12345"}` where only `id` varies, the
+template is `{"request":1,"type":"user_info","id":"{{REQ_BODY_USER_ID}}"}` —
+NOT `{{whole_body}}`, which throws away the ability to pin down everything else.
+Then:
+
+1. In the claim you build for `requestClaim`, set
+   `bodySniff: { enabled: true, template: '<that exact string>' }` and add a
+   matching witness/extracted param `REQ_BODY_USER_ID: <the real id>`, so the
+   real value stays independently verifiable while staying out of the hash.
+2. In the corresponding `allowedInjectedRequestData` / `allowedJsRequests`
+   entry, set `body` to that EXACT SAME literal template, byte for byte — not a
+   `${var}` substitution.
+3. Check by hand that
+   `template.replace('{{REQ_BODY_USER_ID}}', '<a real sample id>')` reproduces
+   the real request body byte for byte. Extra whitespace, different key order,
+   or a field that also varies but wasn't templated → the hashes won't match.
+   Multiple varying fields → one `{{placeholder}}` and one witness param each;
+   you're not limited to one hole.
+
+**Fallback — whole-body placeholder.** If precise per-field templating keeps
+mismatching and you can't pin every variable field down, template the ENTIRE
+body as `{{REQ_BODY_GRD}}`. The `GRD` suffix matters: per the template engine
+(`convertTemplateToRegex`), a placeholder name ending in `GRD` becomes a GREEDY
+match (`(.*)`, including embedded delimiters) instead of the default non-greedy
+`(.*?)`, which is what you want for a placeholder covering the whole remaining
+body. Strictly a fallback — nothing about the body's structure is pinned by the
+hash anymore (any body at all hashes the same).
+
+## How `verifyProof(proof, { providerId })` gates
+
+Traced from `@reclaimprotocol/js-sdk`
+(`src/utils/proofValidationUtils.ts`, `src/utils/providerUtils.ts`,
+`src/witness.ts`):
+
+1. `assertValidateProof` → `fetchProviderHashRequirementsBy(providerId, …)`
+   fetches the provider's CURRENT PUBLISHED CONFIG and builds a list of expected
+   hashes — one per `requestData` entry PLUS one per
+   `allowedInjectedRequestData` (old-devtools) / `allowedJsRequests` (builder)
+   entry.
+2. Each expected hash is `hashRequestSpec` → `hashProofClaimParams({ url,
+   method, body, responseMatches, responseRedactions })` — the same opaque
+   hashing as above, applied to the whole spec.
+3. The real submitted proof's `claimData.parameters` is hashed the same way,
+   with `isOptional`-driven 2^N combinatorial expansion over
+   `responseMatches`/`responseRedactions` PAIRS
+   (`getProviderParamsAsCanonicalizedString`) so an optional field can be
+   present or absent. **That expansion only touches match/redaction pairs, never
+   `url`.**
+4. `assertValidProofsByHash` requires the real proof's hash to equal exactly ONE
+   expected hash. No match → `verifyProof` rejects, even for a perfectly valid,
+   correctly signed proof whose `responseMatches` are 100% right.
+
+### `url` is hashed VERBATIM — zero template/regex resolution
+
+Even with `urlType: REGEX` or `TEMPLATE`. Confirmed from
+`generateSpecsFromRequestSpecTemplate` (the function that DOES resolve `${var}`
+in matches/redactions): it spreads every other field through unchanged and never
+touches `url`. `urlType` is consumed by a different validation path entirely
+(matching a live request against the spec at capture time), never by this hash.
+
+**So, before populating `allowedInjectedRequestData`/`allowedJsRequests` for a
+`verifyProof({ providerId })` use case:** if the real captured URL contains ANY
+component that varies between two otherwise-identical sessions — a tracking
+nonce, a CSRF token, a cache-busting timestamp, a per-request trace id — the
+hash from your declared spec can NEVER equal any real future proof's, no matter
+how correct everything else is. This is common on JS-heavy sites (Slack
+regenerates `_x_id`, `_x_csid` and `fp` on every API call, so a captured
+`users.profile.getSections` URL is never identical twice, even seconds apart).
+
+Populating the field for such a provider is **actively worse than leaving it
+empty**: it converts "no provider-hash check was attempted" into
+"`verifyProof({ providerId })` rejects every real proof, unconditionally."
+
+**Verify first:** capture the same endpoint twice in two separate sessions
+(ideally two users, or the same user on two days) and diff the full URL byte for
+byte.
+
+- Identical every time → safe to populate (`urlType: CONSTANT`, literal URL).
+- Not identical → do not populate it expecting it to gate `verifyProof`. No
+  amount of care with `responseMatches`/`responseRedactions`/`body` fixes it on
+  either backend; it's a property of the hashing itself.
+
+### Optional fields
+
+Independent of the `url` constraint: even with a stable URL,
+`responseMatches`/`responseRedactions` must match EXACTLY, not just in spirit. A
+provider whose extracted field SET varies per user (optional profile fields
+present for some, absent for others) needs every such field marked
+`isOptional: true` on BOTH the match and its paired redaction, so the 2^N
+expansion produces a valid hash for the "absent" case. A sometimes-present field
+that isn't marked optional breaks the hash for every user missing it — the same
+failure mode as the `url` constraint, scoped to one field.
+
+### Fallback
+
+Whether it's an unstable `url`, an unmarked optional field, or any other
+persistent mismatch, the way out is on the VERIFYING side:
+`verifyProof(proof, { dangerouslyDisableContentValidation: true })`. That skips
+steps 1–4 entirely and trusts only the attestor's cryptographic signature; it
+does NOT check that the proof's url/method/body/matches/redactions match
+anything you declared. A caller-supplied `{ hashes: [...] }` computed some other
+way is the alternative when you still want hash-equality enforced, just not
+derived from the published config. Reach for
+`dangerouslyDisableContentValidation` deliberately, never as a silent default —
+it trades away the guarantee that the proof is shaped the way the provider's
+spec says.
+
+## Related topics
+
+- `user-script` — `allowedJsRequests` / `allowedInjectedRequestData`, the fields
+  this page's `url` constraint applies to.
+- `prove` — `run_proof` and diagnosing a proof that fails before it ever reaches
+  `verifyProof`.

package/skill/interception.md

@@ -0,0 +1,236 @@
+# Live interception: `injectionType` / `interceptorType`
+
+## Contents
+
+- The field, per backend
+- Which mechanism to pick, and when to use any of them at all
+- When a site won't load, or won't let the user sign in — ask which clients
+  ship it, bisect, then the remedies
+- The document-replay flag, and why the two backends invert its polarity
+- Recognizing live interception on a provider you didn't author
+- Related topics
+
+Separate from the user script's CONTENT is HOW the client captures network
+traffic for it to consume. This is a per-version field alongside
+`customInjection`/`jsUserScripts`, not something you write into the script.
+
+Whichever mechanism is selected, the script consumes what was caught through the
+SAME API: `window.reclaimInterceptor.addResponseMiddleware(middleware,
+middlewareName?)`. That fires for any intercepted request under EVERY mechanism
+below, `CDP` included — it is not HAWKEYE-specific.
+
+## The field, per backend
+
+- **old-devtools:** `injectionType`, one of
+  `NONE | MSWJS | XHOOK | CDP | HAWKEYE`. The backend's own column default is
+  **`HAWKEYE`, not `NONE`** — harmless for the standard flow, since with no
+  interceptor script installed nothing calls `addResponseMiddleware`.
+  `create_provider_version_from_capture` exposes it as an author-settable input
+  with NO forced default: omit it and the backend default applies; pass an
+  explicit value (including `NONE`) for certainty. When passed it's sent on both
+  register and add-version, and the tool's always-on add-version follow-up (see
+  `disableRequestReplay` below) reapplies it, so an explicit value is guaranteed
+  correct by the time that version lands.
+- **builder:** `webSettings.inapp.interceptorOptions.interceptorType`, one of
+  `HAWKEYE | MSWJS | CDP`. There is no `NONE` here — **omitting
+  `interceptorOptions` entirely IS builder's no-interception signal**, and the
+  publish tool omits `inapp` rather than defaulting to `HAWKEYE`.
+
+Don't assume a non-`NONE` `injectionType` on an existing provider means
+something is broken — HAWKEYE-by-default is normal and inert for the standard
+flow.
+
+## Which mechanism, and when to use any of them at all
+
+- **`CDP`** — interception at the browser/DevTools-protocol level, below the
+  page's own JS. Works uniformly for every request the page makes, including its
+  own document/navigation load. No caveats; works best.
+- **`HAWKEYE` / `MSWJS` / `XHOOK`** — interception inside the page's JS context
+  (different libraries, same idea). Fine for ordinary sub-resource/API requests,
+  but the page's OWN document request is a special case — see
+  "The document-replay flag".
+- **Reach for any of them only when the data genuinely can't be reproduced by
+  an independent replay** — for example, it depends on one-time server-side
+  state (a nonce, a websocket push) that asking again wouldn't return. For
+  everything else the standard capture → replay flow is simpler, and is what
+  `propose_provider` / `run_proof` are built for.
+
+## When a site won't load, or won't let the user sign in
+
+Three symptoms point the same way:
+
+- the page loads blank, half-rendered, or spins forever;
+- sign-in is refused, or the form submits and bounces straight back;
+- login succeeds, then the next page breaks.
+
+### First, ask which clients ship this provider
+
+**Ask the author which verification clients their END USERS will run — don't
+infer it.** It decides whether any of this section applies, and it is not
+something the session can tell you: the client you author through is `builder`,
+whereas production is whatever the author's own app integrates. A provider that
+looks healthy here can fail for every real user, and the reverse.
+
+Ask plainly, and offer the list: *"Which clients will run this in production —
+the InApp SDK, the verifier app, the browser extension, portals, or something
+else? I need it to tell whether interception is a plausible cause."* If they
+ship more than one, the diagnosis has to hold for all of them, so treat the
+strictest row below as the answer.
+
+| Client the end user runs | Where it intercepts | Does this section apply? |
+| --- | --- | --- |
+| `inapp-sdk` | Inside the page's JS | **Yes** |
+| `verifier-app` | Inside the page's JS | **Yes** |
+| `reclaim-browser-extension` | Inside the page's JS | **Yes** |
+| `portals` | Browser level, over CDP | No |
+| `builder` | Server-side CDP, out of band | No |
+
+`builder` is the row for authoring and for a session created with no
+`verificationClientUrl`. Reaching it means you are diagnosing the authoring
+tool, not what the author's users will hit — which is exactly why you ask
+first.
+
+**If production is only `portals` or `builder`, stop here and look elsewhere.**
+CDP intercepts below the page's own JavaScript, so there is nothing for a site
+to detect and nothing to disable — and `builder` never delivers intercepted
+traffic to the page at all (topic `user-script`). A verification on those that
+won't load or won't sign in is almost always a CAPTCHA or bot challenge
+instead: check `analyze_request_constraints` for a `botChallenge` concern, and
+treat it as a site that needs a human through a live view rather than a
+provider to fix.
+
+### Then bisect, cheapest change first
+
+**For the three in-page clients, suspect interception before the recipe.** A
+JS-level interceptor patches the page's own `fetch`/`XMLHttpRequest`, and some
+sites treat that as tampering: anti-bot vendors probe for it directly (a
+patched `fetch.toString()` is a well-known signal), and a few break on their
+own without meaning to. None of that shows up while you author — the MCP agent
+captures over CDP, out of band, so the site behaves normally right up until a
+real verification runs.
+
+Try document replay first and interception second. Often it is only
+DOCUMENT REPLAY that breaks a site, not interception as a whole — the page's
+own top-level navigation is the one request pulled through the interceptor, and
+that is the one some sites refuse to survive. Turning it off is a much smaller
+change than going interceptor-free, and it tells you which half is at fault.
+Copy this checklist and work down it:
+
+- [ ] 1. **Turn document replay OFF, keep the interceptor.** Old-devtools:
+      `disableRequestReplay: true`. Builder:
+      `interceptorOptions.isDocumentRequestReplayEnabled: false` (the two
+      backends name this with opposite polarity — see "The document-replay
+      flag"). Both publish tools already default it off, so a provider that
+      breaks here usually had it switched ON deliberately.
+- [ ] 2. **Retry the verification.** Reload, then sign in as a real end-user
+      would. Works now → stop here. This is the cheapest fix and you keep the
+      interceptor.
+- [ ] 3. **Still broken? Turn interception off entirely.** Old-devtools:
+      publish with `injectionType: NONE`. Builder: omit `interceptorOptions`
+      entirely — there is no `NONE` there, and omitting it IS the off switch.
+- [ ] 4. **Retry again, and compare.** Loads and signs in now → interception
+      itself was the cause; pick a remedy below. Still broken → interception
+      is innocent; go back to the recipe (topic `prove`) or the browser
+      (topic `browser`).
+
+If step 2 fixed it, you are done — the interceptor stays and only the document
+request stops being replayed. Note that under `CDP` the flag is inert, so a
+`CDP` provider that breaks is never a document-replay problem.
+
+### Remedies
+
+If it took step 3, take the first remedy that fits — in this order:
+
+1. **Try a different interceptor.** They patch the page in different ways, so
+   a site that rejects one often accepts another — work along the row for your
+   backend before giving up on interception:
+   - old-devtools: `HAWKEYE` → `MSWJS` → `XHOOK` → `CDP`
+   - builder: `HAWKEYE` → `MSWJS` → `CDP`
+
+   `CDP` is the one that cannot be detected — it intercepts below the page's
+   JavaScript, so nothing in the page is patched. Prefer it when the target
+   client supports it; the in-page clients above may not, which is why the
+   JS-level mechanisms are worth cycling through first.
+2. **Ship without it.** Most providers never needed an interceptor: the
+   standard capture-and-replay flow proves the data on its own. Leave it off
+   unless the value genuinely can't be reproduced by an independent replay
+   (see "Which mechanism, and when to use any of them at all").
+3. **Drive the claim yourself.** Where the data only exists in a live page,
+   skip `reclaimInterceptor` and have the user script navigate to the page
+   that shows the value, read what the page already holds, and call
+   `window.Reclaim.requestClaim` with it. Declare anything the script fires in
+   `allowedJsRequests` / `allowedInjectedRequestData`, and read topic
+   `hash-validation` first — a session-unique URL there makes
+   `verifyProof({ providerId })` reject every real proof.
+
+⚠️ **Remedy 3 needs a client that consumes `window.Reclaim`.** The InApp
+SDK, the browser extension, and the portals do. Builder's own verification
+client injects the bridge but never drains it, so `requestClaim` is inert
+there — see topic `user-script`. Don't reach for remedy 3 on a Builder-run
+verification.
+
+## The document-replay flag
+
+**Both publish tools default this to disabled, and both let the author override
+it.**
+
+- **old-devtools:** `disableRequestReplay`, schema default `false`. The publish
+  tool sends `true` by default, auto-following a fresh register with an
+  add-version call to apply it (the register endpoint hardcodes it `false`
+  regardless of what's sent). Pass `disableRequestReplay: false` explicitly for
+  real document replay.
+- **builder:** `webSettings.clientOptions.inapp.interceptorOptions.isDocumentRequestReplayEnabled`
+  — positive naming, schema default `false`. The publish tool sets `false`
+  whenever you pass `interceptorOptions` at all. Pass
+  `interceptorOptions.isDocumentRequestReplayEnabled: true` to enable it.
+
+The two backends' polarity is inverted, so don't carry an assumption across.
+The old dashboard makes this worse: its checkbox is labeled "Enable Document
+Request Replay" and is CHECKED when `disableRequestReplay` is `false`. Its field
+hint, verbatim: *"When enabled (default), the main document HTTP request is
+actually sent to get the response. When disabled, the expected request and
+response are constructed locally without making the network call."*
+
+**This does NOT gate `addResponseMiddleware` in general** — that fires for any
+intercepted request regardless. What it actually controls is whether the CURRENT
+PAGE'S OWN document/navigation request (the one that loads the page's HTML, not
+a sub-resource or API call) is sent for real over the network, for the non-CDP
+mechanisms. It only matters when the request you're proving *is* that document
+request — that is, when the target value is embedded in the page's own HTML
+rather than fetched by a separate API call.
+
+- Document request genuinely sent → `addResponseMiddleware` sees it like any
+  other request. This is what you need to prove data straight out of the page's
+  HTML.
+- Document request constructed locally → the interceptor never observes it
+  (every OTHER request is still intercepted normally). This is a **per-site
+  compatibility escape hatch**: some sites misbehave when their top-level
+  navigation is pulled through a JS-level interceptor. Not a privacy or
+  architecture choice.
+- **`CDP` is exempt from all of it** — it intercepts at the browser level, so
+  the document request is never a special case and this flag is inert.
+
+## Recognizing this on a provider you didn't author
+
+`get_provider_info` (old-devtools mode; reads the public SDK backend) returns
+the raw `providerConfig`. If `injectionType`/
+`interceptorType` is set to something other than the inert default AND the
+`customInjection`/`jsUserScripts` calls `reclaimInterceptor.addResponseMiddleware`
+(or references `MSWJS`/`XHook`), that provider is on the live-interception path,
+not the standard replay flow — read the script before carrying it forward or
+editing around it.
+
+If, on top of that, the request being proven IS the login/landing page itself
+(its `url` matches `loginUrl`, so it's the document request) and the mechanism
+isn't `CDP`, check the document-replay flag too: a value that routes it around
+live interception was a deliberate site-compatibility decision, not an
+oversight.
+
+## Related topics
+
+- `user-script` — what the script itself does with intercepted traffic, and why
+  `window.Reclaim` is inert on Builder's own client.
+- `hash-validation` — declaring script-fired requests without breaking
+  `verifyProof({ providerId })`.
+- `browser` — the modes with a live view (`container`, `builder`), for handing
+  a stuck sign-in to a human.

package/skill/js-sdk-integration.md

@@ -3,10 +3,16 @@
 Keep `appSecret` on your BACKEND only. Everything below runs server-side;
 the browser only ever sees the `requestUrl`.
 
+## Contents
+
+- Testing locally, without deploying anything
+- Steps 1-4: install, create the proof request, open it, verify the callback
+- Gotchas
+
 ## Testing locally, without deploying anything
 
 - **providerId `"example"`**: a canonical built-in smoke-test provider —
-  no need to author/publish your own provider just to confirm the SDK
+  no need to author and publish your own provider to confirm the SDK
   plumbing works end-to-end. Swap in your real providerId once this passes.
 - **A public callback URL**: `setAppCallbackUrl` needs a URL Reclaim's
   servers can reach over the public internet — `localhost` will never
@@ -15,9 +21,9 @@ the browser only ever sees the `requestUrl`.
     alongside your local server, then set `PUBLIC_URL` to the
     `https://<id>.ngrok-free.app` URL it prints. Zero-deploy, but the tunnel
     URL is temporary.
-  - **A dev/staging server you can deploy to quickly**: if you already have
-    one reachable from the internet, just point `PUBLIC_URL` at it instead —
-    skips the extra local tool.
+  - **A development or staging server you can deploy to quickly**: if you
+    already have one reachable from the internet, point `PUBLIC_URL` at it
+    instead — that skips the extra local tool.
   Without either, only the frontend-side flow (opening `requestUrl` and
   reading the result client-side) works — the async webhook never lands.
 - `scaffold_reclaim_demo_server` defaults `providerId` to `"example"`; its
@@ -63,9 +69,9 @@ reclaimProofRequest.setContext(
 const requestUrl = await reclaimProofRequest.getRequestUrl();
 
 // Hand the requestUrl to the frontend to open in a new tab/window/webview.
-// If you need to rehydrate this request object later (e.g. in the webhook
-// handler, in a different process), persist reclaimProofRequest.toJsonString()
-// (sync) and rehydrate with:
+// If you need to rehydrate this request object later (for example, in the
+// webhook handler, in a different process), persist
+// reclaimProofRequest.toJsonString() (sync) and rehydrate with:
 //   const req = await ReclaimProofRequest.fromJsonString(json); // ASYNC
 ```
 

package/skill/old-mode.md

@@ -1,21 +1,62 @@
-This server is running against the devtools backend, so its capabilities are intentionally limited.
-
-**Provider-authoring flow:**
-
-1. `reclaim_authenticate` — by default opens the dashboard in Chrome, waits for you to sign in, captures the token automatically, and closes the tab (or paste a `token` / pass an eth `ethAddress` to skip the browser). Do this once before publishing or listing. (Note: this old-devtools login is separate from `authenticate_builder`, which is needed only for the two Builder-exclusive features below — remote-browser allocation and org listing — see the auth troubleshooting note below.)
-2. `attach_browser` → `start_capture` → drive the page (`navigate`, `wait_for_page`) → inspect (`list_requests`, `get_request`, `find_requests_containing`). Browser source is independent of this publishing mode: local Chrome works as before; `builder` uses `authenticate_builder` + `list_builder_organizations` to allocate a quota-accounted Popcorn browser and returns a shareable live-view URL; `custom` accepts a caller-supplied CDP websocket.
-3. `propose_provider` to draft a provider, `replay_request` to pre-flight, `run_proof` to verify with the attestor. OPRF/`hash` redactions are optional and supported. When the data spans MULTIPLE requests (e.g. a landing call + a details call), propose + `run_proof` EACH request to get one proven draft per request.
-4. `create_provider_version_from_capture` to publish. With NO `providerId` it REGISTERS A NEW provider (re-running creates a duplicate); pass a `providerId` from `get_me_providers` to add a new version to an existing provider instead. For a MULTI-REQUEST provider, pass `providers: [draftA, draftB, …]` (every proven draft) so EACH becomes a parallel `requestData` entry on the one version — passing a single `provider` saves only that one request.
-5. `get_me_providers` lists the providers you own (the only provider-listing tool available in this mode).
-
-**Editing / new versions (ownership):** you can only add a version to a provider **you own** — the `providerId` must come from `get_me_providers`. Trying to version a public provider you don't own fails with a permission error; that is expected. To publish your own version of such a site, re-run `create_provider_version_from_capture` **without** `providerId` to register your own copy (a separate provider you own). If versioning a provider you *do* own fails, confirm you authenticated as the account that created it.
-
-**Two independent auth systems — don't confuse them:** `authenticate_builder` and `reclaim_authenticate` log into DIFFERENT backends and are NOT interchangeable. In THIS mode (old-devtools), `authenticate_builder` is needed only for two Builder-exclusive features — allocating a Popcorn remote browser (`attach_browser mode="builder"`) and listing Builder organizations (`list_builder_organizations`, since old-devtools has no generated org tool of its own). It does not authorize `create_provider_version_from_capture`, `get_me_providers`, or any other old-devtools call — those need their own `reclaim_authenticate` login. (Builder mode, by contrast, needs no separate login at all — its one native auth tool covers everything, org listing included via `list_orgs`.) `get_devtools_mode` reports the presence of both. If a publish or `get_me_providers` call fails with an auth error even though you (or the dev) already ran `reclaim_authenticate` once before, the old-devtools token has likely expired (Firebase tokens are short-lived, ~1h) — re-run `reclaim_authenticate` and retry; don't assume `authenticate_builder` fixes it.
-
-Only GET/POST endpoints can be captured into providers here. There is no Builder organization-management, credential-linking, or public-review workflow in this mode; the narrow `authenticate_builder` and `list_builder_organizations` exception exists only to allocate Builder-hosted authoring browsers.
-
-**Diagnosing a session:** `session_analytics_logs` fetches the analytics logs for a verification session by id (from the logs service) — use it to see what happened in a session (matches, claims, failures).
-
-**Templating:** bare `{{name}}` placeholders are provider params (values extracted from the response). Values the CONSUMER must supply at verification time (e.g. a username in the URL path) use the `{{context_<name>}}` form — this backend does not support the dotted `{{context.<name>}}` interpolation the new builder uses.
-
-**Backend mode:** this is the reduced devtools surface — the BUILDER backend (generated API tools, orgs, verification sessions, public review) is one switch away. `get_devtools_mode` reports the active mode and why it's in effect; `set_devtools_mode` persists a switch to `~/.reclaim/config.json` — it takes effect on the NEXT server start (restart/reconnect), and an explicit `USE_OLD_DEVTOOLS` env var overrides the persisted choice.
+Reclaim provider-authoring tools, running against the OLD DEVTOOLS backend —
+intentionally reduced: no generated API tools, no organizations, no public
+review flow. Only GET/POST requests can be captured into providers.
+
+**A `provider` is a recipe for extracting a data point from a site. A
+`verification` is an end-user session that proves a data point using one. Never
+call a provider a "verification."**
+
+**Read the guide before authoring.** `how_it_works({ topic })` serves it, one
+topic at a time — call `how_it_works({})` for the topic index. Topics:
+`overview`, `auth`, `browser`, `capture`, `draft`, `params`, `prove`, `publish`,
+`user-script`, `interception`, `hash-validation`, `credentials`, `never-do`,
+`troubleshooting`.
+
+**Ask who is doing the signing in, then offer a browser.** "Yourself, or does
+someone else need to log in?" decides half of `attach_browser`'s list. Doing it
+themselves → `dedicated` or `attach`, both local, neither shareable. Someone
+else, or a phone-only site → `container` (the Reclaim runtime in Docker, free,
+shareable with `share_browser_view`) or `builder` (hosted, no install,
+**chargeable**). Say which costs money, and say `container` exists at all:
+nobody guesses it.
+
+**The flow:** `reclaim_authenticate` once → `attach_browser` (ask the developer
+which browser source — local Chrome, the Reclaim runtime in Docker, a remote
+Builder browser, or a CDP URL they supply) → `start_capture` ⇒ **`captureId`** →
+drive the page (`navigate`,
+`wait_for_page`, `eval_in_page`) → inspect (`list_requests`, `get_request`,
+`find_requests_containing`) → `propose_provider` ⇒ **`draftId`** →
+`replay_request(captureId, draftId)` → `run_proof(captureId, draftId,
+ownerAddress)` → `create_provider_version_from_capture`.
+
+**Publishing and ownership:** with NO `providerId`, publish REGISTERS A NEW
+provider — re-running creates a duplicate. Pass a `providerId` from
+`get_me_providers` (the only provider-listing tool here) to add a version to an
+existing provider instead; you can only version a provider you own. For a
+multi-request provider pass `providers: [draftA, draftB, …]` so each becomes a
+parallel `requestData` entry on the one version.
+
+**Two independent auth systems — don't confuse them.** `reclaim_authenticate`
+authorizes everything in this mode; its Firebase token is short-lived (~1h), so
+an auth error on publish or `get_me_providers` means re-run it.
+`authenticate_builder` covers only two Builder-exclusive features: allocating a
+remote browser (`attach_browser mode="builder"`) and
+`list_builder_organizations`. It authorizes nothing else. `get_devtools_mode`
+reports both.
+
+**Never prove a public endpoint.** After `replay_request` succeeds, re-run it
+with `withoutSecrets: true`. If the value still extracts, the endpoint is public
+and the proof binds to nobody — find an auth-bound request instead.
+
+**Templating:** bare `{{name}}` placeholders are extracted from the response.
+Values the CONSUMER supplies at verification time use `{{context_<name>}}` —
+this backend does NOT support the builder's dotted `{{context.<name>}}` form.
+
+**Diagnosing a session:** `session_analytics_logs` fetches a verification
+session's analytics logs by id (matches, claims, failures).
+
+**Backend mode:** the BUILDER backend (generated API tools, orgs, verification
+sessions, public review) is one switch away. `get_devtools_mode` reports the
+active mode and why; `set_devtools_mode` persists a switch to
+`~/.reclaim/config.json`, effective on the NEXT server start. An explicit
+`USE_OLD_DEVTOOLS` env var wins over the persisted choice.