@reclaimprotocol/agent 0.1.23 → 0.1.24

package/skill/injection-api.md

@@ -1,364 +0,0 @@
-# Page injection: `jsUserScripts` / `customInjection`
-
-A provider can inject **plain JavaScript** that runs in the page before every
-page load. Builder stores it as `webSettings.jsUserScripts`; old-devtools stores
-it as `customInjection` (identical semantics). It is ordinary browser
-JavaScript — its job is to drive the page so the target API request(s) fire
-during a verification.
-
-## What the script is for
-
-The verification client already **intercepts the page's network requests
-automatically** — the script does NOT necessarily ned to capture, forward, or "claim" them.
-Its primary job is to get the page into the state where those requests happen, using
-standard browser APIs:
-
-- **Navigate** with `window.location.href = '…'` / `location.assign(…)` / the
-  history API.
-- **Interact**: select an element and `.click()` it, submit a form, dispatch
-  events.
-- **Wait**: poll the DOM until login/mfa completes or content loads, then trigger
-  the next step.
-
-Write it like any page script. It runs before EVERY page load, so keep it
-idempotent, cheap, and guarded against running at the wrong time.
-
-## `window.Reclaim` — rarely needed, use sparingly
-
-`window.Reclaim` is an OPTIONAL bridge for a few edge cases only. Most providers
-never touch it. In particular it does **not** do navigation — use the standard
-browser APIs above for that, in plain `jsUserScripts` / `customInjection`.
-
-Do not reach for `window.Reclaim` by default. Only use it when an edge case
-genuinely requires it. Its full, authoritative API (every method, with exact
-signatures and semantics) is documented in 
-`https://github.com/reclaimprotocol/provider-web-script-dev/blob/main/src/support/env.d.ts` — consult that file rather
-than guessing.
-
-> Both backends run the script: the Builder verification client installs
-> `jsUserScripts` to run before every page load (before navigating to the
-> provider's `initialUrl`), and old-devtools runs `customInjection` the same
-> way. (`window.Reclaim` itself is still only forwarded/available in the cases
-> noted above.)
-
-## Interception mode: `injectionType` / `interceptorType`
-
-Separate from the script's CONTENT is HOW the client captures network traffic
-for it to consume — a per-version field alongside `customInjection`/
-`jsUserScripts`, not something you write into the script itself. Whichever
-mechanism is selected, the injected script consumes what it caught through
-the SAME API: `window.reclaimInterceptor.addResponseMiddleware(middleware,
-middlewareName?)`. That consumption API fires for any intercepted request
-under EVERY mechanism below, including `CDP` — it is not a HAWKEYE-specific
-thing.
-
-- **old-devtools** field: `injectionType`, one of `NONE | MSWJS | XHOOK | CDP |
-  HAWKEYE` — confirmed the literal field name against a real snapshot of
-  legacy-backend provider data (`packages/app/scripts/data/
-  provider-configs.snapshot.json`, captured straight from the old-devtools
-  Mongo `Provider`/`ProviderVersion.providerConfig`). The backend's own
-  column default is `HAWKEYE`, not `NONE` — harmless for the standard flow
-  (no interceptor script installed, so nothing calls
-  `addResponseMiddleware`). `create_provider_version_from_capture`
-  (old-devtools mode) now exposes `injectionType` as an author-settable
-  input, with NO forced default — omit it and the backend's own `HAWKEYE`
-  default applies, same as before this field was exposed at all; pass an
-  explicit value (including `NONE`) if you want certainty either way. When
-  passed, it's sent on both register and add-version; since it's unverified
-  whether the register controller actually persists it (no source access to
-  confirm), the tool's existing always-on add-version follow-up (see
-  `disableRequestReplay` below) reapplies it regardless, so an explicit value
-  is guaranteed correct by the time the follow-up version lands.
-- **builder** field: `webSettings.inapp.interceptorOptions.interceptorType`,
-  one of `HAWKEYE | MSWJS | CDP` (no `NONE`/`XHOOK` in this schema — omitting
-  `interceptorOptions` entirely is builder's NONE-equivalent).
-  `create_provider_version_from_capture` (builder mode) exposes
-  `interceptorOptions.interceptorType` as an author-settable input; passing
-  it is how you opt INTO real interception. Omit it (the default) for the
-  standard flow, and the tool correctly omits `inapp` — it does NOT default
-  to `HAWKEYE` (that was a bug in an earlier version of this tool; fixed).
-
-**The mechanisms, and why you'd pick one over plain replay:**
-- **`CDP`** — interception at the browser/DevTools-protocol level, below the
-  page's own JS. Works uniformly for every request the page makes, INCLUDING
-  its own document/navigation load — no caveats, "works best."
-- **`HAWKEYE` / `MSWJS` / `XHOOK`** — interception done IN the page's JS
-  context (different underlying libraries; same idea). These see the page's
-  ordinary sub-resource/API requests fine, but the page's OWN document
-  request is a special case — see `disableRequestReplay` below.
-- Reach for ANY of the above (over the plain replay flow, `injectionType:
-  NONE` / no `inapp`) only when the data genuinely can't be reproduced by an
-  independent replay — e.g. it depends on one-time server-side state (a
-  nonce, a websocket push) that asking again wouldn't give you back. For
-  everything else, the standard flow is simpler and is what `propose_provider`
-  / `run_proof` are built for.
-
-## `disableRequestReplay` / `isDocumentRequestReplayEnabled`
-
-**`create_provider_version_from_capture` always force-disables this now, in
-BOTH backends** — old-devtools always publishes `disableRequestReplay: true`
-(the tool auto-follows a fresh register with an add-version call to apply
-it, since the register endpoint hardcodes it `false` regardless of what's
-sent). Builder always publishes `isDocumentRequestReplayEnabled: false`
-under `webSettings.clientOptions.inapp.interceptorOptions` **whenever you
-pass `interceptorOptions` at all** — but if you don't (the standard,
-non-interception flow), the tool omits `inapp` entirely rather than forcing
-a default `interceptorType`, because builder's schema has no `NONE`
-interceptorType: omitting the whole object IS its no-interception signal,
-and defaulting it to (even inert) `HAWKEYE` would misrepresent the
-provider. There is no author-facing way to turn document replay back on
-through either tool once `interceptorOptions`/`injectionType` is in play.
-The rest of this section explains the underlying platform semantics for
-context (e.g. reading an existing provider someone else published outside
-these tools) — the replay sub-flag itself is no longer something you
-configure here.
-
-**This does NOT gate `addResponseMiddleware` in general** — that fires for
-any intercepted request regardless of this field. What it actually controls:
-whether the CURRENT PAGE'S OWN document/navigation request (the request that
-loads the page's HTML — not a sub-resource or API call) is sent for real
-over the network at all, for the non-CDP mechanisms above. And it only
-matters if the request you're proving *is* that document request — e.g. the
-target value is embedded straight in the page's own HTML rather than
-fetched via a separate API call.
-
-- `false` (old-devtools' schema default) — the document request IS sent for
-  real (for `HAWKEYE`/`MSWJS`/`XHOOK`), so `addResponseMiddleware` sees it
-  like any other intercepted request. This is what you need if you're
-  proving data straight out of the current page's own HTML.
-- `true` — the document request is constructed/synthesized locally instead
-  of actually being sent, so the interceptor never observes it (every OTHER
-  request the page makes is still intercepted normally). This exists as a
-  **compatibility escape hatch**: some websites misbehave when their own
-  top-level navigation request gets pulled through a JS-level interceptor —
-  flip this to `true` for those sites. It's not a privacy or architecture
-  choice, it's a per-site workaround.
-- **`CDP` is exempt from all of this** — it intercepts at the browser level,
-  so the document request is never a special case for it.
-  `disableRequestReplay`/`isDocumentRequestReplayEnabled` is simply inert
-  when `injectionType`/`interceptorType` is `CDP`.
-
-Field names, per backend:
-- **old-devtools**: `disableRequestReplay` (boolean, schema default `false`).
-  The dashboard's own field hint (verbatim): *"When enabled (default), the
-  main document HTTP request is actually sent to get the response. When
-  disabled, the expected request and response are constructed locally
-  without making the network call."* Note the double negative: the
-  dashboard's checkbox is labeled "Enable Document Request Replay" and is
-  CHECKED when `disableRequestReplay` is `false` — the checkbox and the field
-  it writes are inverted from each other.
-- **builder**: `webSettings.inapp.interceptorOptions.isDocumentRequestReplayEnabled`
-  (boolean, positive naming this time — `true` means the document request IS
-  sent). Per its openapi description, the schema's own stated default is
-  `false`. Don't assume the two backends' defaults line up just because the
-  concept is the same — old effectively defaults to "sent", the builder
-  schema's stated default is "not sent"; verify against whichever backend
-  you're publishing to rather than carrying an assumption across.
-
-There is nothing to configure here through `create_provider_version_from_capture`
-any more (see the note at the top of this section) — both backends always
-publish with document replay disabled.
-
-## `bodySniff` — unrelated to the above
-
-A per-request field (sits next to `url`/`method` on each `requestData` entry,
-not a version-level setting): `{ enabled: boolean, template: string }`.
-
-**`bodySniff.enabled` should be `true` whenever the request HAS a body at
-all** — this isn't a special case reserved for dynamic bodies. Only skip it
-(leave `enabled: false` / template empty) when the request genuinely sends
-no body (typically a GET). Both publish tools in this package already set
-`enabled`/`template` for you from `provider.body` for a normal
-capture-and-replay provider — you don't need to touch it directly there.
-Read the rest of this section if you're authoring a claim via live
-interception (`window.Reclaim.requestClaim`) whose body isn't simply the
-one literal string you captured once — most commonly because some SMALL
-part of it is dynamic per user/run (e.g. a user id, not necessarily the
-whole payload).
-
-**What actually gets hashed.** Confirmed directly from attestor-core
-(`hashProviderParams` in `@reclaimprotocol/attestor-core`): the claim/spec
-hash is `keccak256(canonicalStringify({ url, method, body, responseMatches,
-responseRedactions }))`, where `body` is whatever plain string sits in
-`params.body` — attestor-core does **no** template resolution, substitution,
-or `{{}}`-stripping on it; it hashes exactly the string it's given, verbatim.
-`bodySniff` as a concept (the `{enabled, template}` wrapper) doesn't exist
-inside attestor-core at all — it's translated to a plain `body: string`
-one layer up (old-devtools' `providerUtils.ts`: `body: bodySniff.enabled ?
-bodySniff.template : ''`), and THAT literal string — including any
-unresolved `{{paramName}}` text still in it — is what reaches
-`hashProviderParams`.
-
-**The consequence — and why this differs from `templateParams`.** A live
-claim's own `body` field (submitted via `requestClaim`) can ALSO be set to
-a literal template string containing `{{paramName}}`, rather than the real
-substituted content, with the real value supplied separately as a witness
-param. Since attestor-core hashes `body` as an opaque string with no
-resolution, a claim whose `body` is the literal template `T` and a spec
-whose `body`/`bodySniff.template` is that SAME literal template `T` hash
-identically — regardless of what real value the `{{paramName}}` hole stood
-for. This is the OPPOSITE of `templateParams`/`${var}` (used in
-`responseMatches`/`responseRedactions`), which — per the JS SDK's
-`generateSpecsFromRequestSpecTemplate` — genuinely gets substituted with a
-real value at verify time before hashing. Don't conflate the two: `${var}`
-in a match/redaction template IS resolved before hashing;  `{{var}}` in a
-body template is NEVER resolved before hashing, on either side.
-
-**Practical use**: check exactly which part of the body actually changes
-across users/runs — usually it's one small field (an id, a token, a short
-list), not the whole payload. Template ONLY that minimal changing part;
-leave everything else as fixed literal text. For a body like
-`{"request":1,"type":"user_info","id":"12345"}` where only `id` varies, the
-template is `{"request":1,"type":"user_info","id":"{{REQ_BODY_USER_ID}}"}`
-— NOT e.g. `{{whole_body}}` wrapping the entire object, which would throw
-away the ability to structurally pin down everything else about the
-request. Then:
-1. In the claim you build for `requestClaim`, set `bodySniff: { enabled:
-   true, template: '{"request":1,"type":"user_info","id":"{{REQ_BODY_USER_ID}}"}' }`
-   and add a matching witness/extracted param
-   `REQ_BODY_USER_ID: theRealUserId` so the real value is still
-   independently verifiable, just not part of the hash.
-2. In the corresponding `allowedInjectedRequestData` entry, set `body` to
-   that EXACT SAME literal template string (byte-for-byte, same
-   `{{REQ_BODY_USER_ID}}` placeholder, same everything else) — NOT a
-   `${var}` substitution, since `body` doesn't go through that substitution
-   step.
-3. Before publishing, sanity-check by hand that
-   `template.replace('{{REQ_BODY_USER_ID}}', 'a real sample id')`
-   reproduces the real captured/expected request body byte-for-byte — if it
-   doesn't (extra whitespace, different key order, a field that turned out
-   to ALSO vary but wasn't templated, etc.), the hashes won't match either,
-   since the SPEC side uses the literal template as-is and the real claim's
-   body must independently be built to match that same shape when the
-   placeholder(s) are filled in. If MULTIPLE fields vary independently,
-   template each one with its own `{{placeholder}}` and its own witness
-   param — you're not limited to one hole per template.
-
-**Fallback — whole-body placeholder.** If precise per-field templating
-keeps causing hash mismatches (e.g. more fields turn out to vary than
-expected, key order isn't stable, whitespace differs run-to-run) and you
-can't pin down every variable field individually, fall back to templating
-the ENTIRE body as one placeholder instead: `{{REQ_BODY_GRD}}`. The `GRD`
-suffix matters — per the response-matching template engine
-(`convertTemplateToRegex`), a placeholder name ending in `GRD` is treated as
-a GREEDY match (`(.*)`, matching everything including embedded delimiters)
-rather than the default non-greedy `(.*?)` (which stops at the first
-occurrence of whatever comes after it in the template) — for a placeholder
-covering the entire remaining body with nothing fixed after it, greedy is
-what you want. This is strictly a fallback: it means NOTHING about the
-body's structure is pinned down by the hash anymore (any body at all would
-hash the same), so prefer the minimal-placeholder approach above whenever
-you can actually identify which fields vary.
-
-The same opaque-string hashing (confirmed directly from the attestor-core
-bundle shipped in this repo's `node_modules`) also applies to fields beyond
-`body`. The caller that invokes it against a live submitted claim to gate
-proof acceptance is `@reclaimprotocol/js-sdk`'s `verifyProof` — the next
-section covers how that gate works and how to configure a provider that
-satisfies it.
-
-## Provider-hash validation (`verifyProof(proof, { providerId })`) — how it gates, and how to configure a provider that passes it
-
-The caller for the hashing described above is `@reclaimprotocol/js-sdk`'s
-`verifyProof`. When called with `{ providerId }` (the SDK's own
-integration guide treats this as the normal case), traced directly from
-that package's source (`src/utils/proofValidationUtils.ts`,
-`src/utils/providerUtils.ts`, `src/witness.ts`):
-
-1. `assertValidateProof` → `fetchProviderHashRequirementsBy(providerId, …)`
-   fetches this provider's CURRENT PUBLISHED CONFIG and builds a list of
-   "expected hashes" — one per entry in `requestData` PLUS one per entry in
-   `allowedInjectedRequestData` (old-devtools) / `allowedJsRequests`
-   (builder) — same concept, different field name per backend; don't
-   assume a doc or error naming one applies verbatim to the other.
-2. Each expected hash is `hashRequestSpec` → `hashProofClaimParams({ url,
-   method, body, responseMatches, responseRedactions })` — the SAME
-   opaque-string hashing described above for `body`, applied to the WHOLE
-   spec at once.
-3. The REAL submitted proof's own `claimData.parameters` is hashed the
-   SAME way, with `isOptional`-driven 2^N combinatorial expansion over
-   `responseMatches`/`responseRedactions` PAIRS so an optional field can be
-   either present or absent (`getProviderParamsAsCanonicalizedString`) —
-   but this expansion ONLY touches match/redaction pairs, never `url`.
-4. `assertValidProofsByHash` requires the real proof's hash to exactly
-   equal ONE expected hash. No match → `verifyProof` rejects, even for a
-   perfectly valid, correctly-signed proof whose `responseMatches` are
-   100% correct in shape.
-
-**Key constraint: `url` is hashed VERBATIM, with ZERO template/regex
-resolution — even when `urlType` is `REGEX` or `TEMPLATE`.** Confirmed
-directly from `generateSpecsFromRequestSpecTemplate` (the function that DOES
-resolve `${var}` placeholders in `responseMatches`/`responseRedactions`
-before hashing): it spreads every other field from the template unchanged,
-and never touches `url`. `urlType` is consumed by a DIFFERENT validation
-path entirely (matching an incoming live request against this spec at
-capture time) — never by this hash computation.
-
-**Check this before populating `allowedInjectedRequestData`/
-`allowedJsRequests` for a `verifyProof({ providerId })` use case:** if the
-real captured request URL contains ANY component that varies between two
-otherwise-identical sessions — a tracking nonce, a CSRF token, a
-cache-busting timestamp, a per-request trace id — the hash computed from
-your declared spec can NEVER equal the hash of any real future proof, no
-matter how correct `responseMatches`/`responseRedactions`/`body` are. This
-is common on JS-heavy sites: e.g. Slack regenerates `_x_id`, `_x_csid`, and
-`fp` on every single API call, so a captured `users.profile.getSections`
-URL is never identical twice, even for the same user seconds apart.
-Populating this field for such a provider is actively worse than leaving
-it empty — it converts "no provider-hash check was attempted" into
-"`verifyProof({ providerId })` rejects every real proof from this
-provider, unconditionally."
-
-**Verify before wiring this up**: capture the same endpoint twice, in two
-separate sessions (ideally two different users, or the same user on two
-different days), and diff the full request URL byte-for-byte.
-- Identical every time → safe to populate with `urlType: CONSTANT` and the
-  literal URL.
-- Not identical → do not populate this field expecting it to gate
-  `verifyProof({ providerId })`. There is no way to make a session-unique
-  URL satisfy this specific mechanism, on either backend — this is a
-  property of the hashing itself, not something fixable by editing
-  `responseMatches`, `responseRedactions`, or `body` more carefully. See
-  the fallback below.
-
-Also note, independent of the `url` constraint above: even with a stable URL,
-`responseMatches`/`responseRedactions` must be an EXACT structural match —
-not just similar in spirit. A provider whose extracted field SET varies
-per user (e.g. optional profile fields present for some users, absent for
-others) needs every such field marked `isOptional: true` on BOTH the match
-and its paired redaction, so the 2^N expansion produces a valid hash for
-the "field absent" case too. A field that's sometimes-present but never
-marked optional breaks the hash for any user missing it — the same failure
-mode as the `url` constraint above, just scoped to one field instead of the
-whole spec.
-
-**Fallback if provider-hash validation keeps causing problems.** Whether
-it's an unstable `url`, an un-marked optional field, or any other
-persistent hash mismatch you can't pin down — the way out on the verifying
-side is `verifyProof(proof, { dangerouslyDisableContentValidation: true })`.
-That skips step 1-4 above entirely and trusts only the attestor's
-cryptographic signature over the proof; it does NOT check that the proof's
-`url`/`method`/`body`/`responseMatches`/`responseRedactions` match anything
-you declared. A caller-supplied `{ hashes: [...] }` computed some other way
-is the alternative when you still want hash-equality enforced, just not
-derived from this provider's published config. Reach for
-`dangerouslyDisableContentValidation` deliberately, not as a silent
-default — it trades away the guarantee that the proof is shaped the way
-this provider's spec says it should be.
-
-## Recognizing this on a provider you didn't author
-
-`get_provider_version_config` (old-devtools) and `get_provider_info` (public
-SDK backend) both return the raw `providerConfig`. If you see
-`injectionType`/`interceptorType` set to anything other than the inert
-default AND a `customInjection`/`jsUserScripts` that calls
-`reclaimInterceptor.addResponseMiddleware` (or references `MSWJS`/`XHook`),
-that provider is on the live-interception path, not the standard replay
-flow — read the script before assuming you can safely carry it forward or
-edit around it. If, on top of that, the request being proven IS the login/
-landing page itself (its `url` matches `loginUrl`, i.e. it's the document
-request, not a separate API call) and `injectionType`/`interceptorType`
-isn't `CDP`, also check `disableRequestReplay`/
-`isDocumentRequestReplayEnabled` — `true` there means that request was
-deliberately routed around live interception for site-compatibility
-reasons, not an oversight.