@astrasyncai/verification-gateway 5.21.1 → 5.22.0

package/CHANGELOG.md

@@ -8,6 +8,22 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 Full architectural context for each release lives in the platform spec at
 `docs/research/adapter-architecture-technical-requirements.md`.
 
+## [5.22.0] — 2026-10-26 — edge verdicts on beacons, WAF bot labels, Akamai and Imperva headers
+
+### Added
+
+- **Edge verdict on beacons.** Edge beacons now carry `edgeDecision` (`allow` | `deny`) and `edgeMode`, decided before the beacon is sent. An enforce-mode deny at the edge is recorded as a deny in Visit Intelligence instead of looking like a pass.
+- **`wafLabelPatterns`** on `PlatformSignatureDef` and **`wafLabels`** on `PlatformDetectionInput`: attribute a request by the bot name a customer's WAF assigned (AWS Bot Control, Akamai Bot Manager, Cloudflare verified-bot category, Imperva client app). Seed patterns are added for Claude, ChatGPT, Gemini, Perplexity, Grok and Meta AI. They only ever match rows read from WAF security logs.
+- **`signature-agent`** is now a platform-signal header, and ChatGPT's Web Bot Auth directory host attributes a signed agent whose user agent is stock Chrome.
+- `deriveConnectionFromHeaders`: `Incap-Client-IP` (Imperva) in the client-IP precedence, and Akamai's `X-Akamai-Edgescape` filling country, region, city, lat/long, time zone, ASN and continent where no direct header supplied them.
+
+## [5.21.2] — 2026-10-25 — xAI Grok and Meta AI crawler signatures
+
+### Added
+
+- Platform signatures for xAI Grok (`GrokBot/1.0`, `xAI-Grok/1.0`, `Grok-DeepSearch/1.0`) and Meta AI (`Meta-WebIndexer`, `Meta-ExternalFetcher`), so these crawlers attribute as named platforms (`grok`, `meta-ai`) instead of unknown. Anchored, versioned tokens only: a stock browser UA never matches.
+- A note in the registry: Meta's Muse and OpenAI's Dots publish no user agent and browse through real Chromium sessions, so they can't be detected by UA. Verified identity is the only reliable signal for them.
+
 ## [5.21.1] — 2026-10-24 — pricingContextFrom exported
 
 ### Added

package/README.md

@@ -1083,15 +1083,27 @@ Both consume the same dashboard-managed policy (`./edge-config`), the same platf
 
 Header-level capture (`ObservedMetadata.headers` etc.) is identical everywhere: **automatic on every verify** for the edge adapters and the `express` / `nextjs` / `mcp` SDK adapters (no config, no opt-out). What differs is the **connection layer** — and the deciding variable is _who operates the CDN in front of your origin_:
 
-| Integration                                                                                        | Connection layer                                                                                                          | TLS fingerprint (JA3/JA4)                                                                                                                                                       |
-| -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **Edge adapter** (`adapter-lambda`, `adapter-edge`)                                                | **Full** — native platform signals (IP, ASN, geo, TLS, HTTP version, device class, header structure)                      | CloudFront: **JA3 + JA4** (2.2.0+, via the template's origin request policy). Cloudflare Workers: Bot Management zones only. Vercel: JA4 with BotID. Fastly: none               |
-| **SDK behind a CDN you control** (your own Cloudflare / Fastly / Vercel / CloudFront distribution) | **Derived** — geo/ASN/TLS promoted from the CDN's injected headers (`deriveConnectionFromHeaders`)                        | Cloudflare: JA3 + JA4 (`cf-ja3-hash` / `cf-ja4`, plan-dependent). CloudFront: JA3 + JA4 — add the `CloudFront-Viewer-JA3/JA4-Fingerprint` headers to your origin request policy |
-| **SDK behind a platform-managed CDN** (Railway, Render, Fly, Heroku, …)                            | **IP only** — platform CDNs forward `x-forwarded-for` but NOT their provider-specific geo/TLS headers to customer origins | None — no fingerprint header reaches your origin                                                                                                                                |
-| **SDK, no CDN**                                                                                    | Absent (IP from `x-forwarded-for` when a proxy sets it)                                                                   | None                                                                                                                                                                            |
+| Integration                                                                                        | Connection layer                                                                                                                 | TLS fingerprint (JA3/JA4)                                                                                                                                                       |
+| -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Edge adapter** (`adapter-lambda`, `adapter-edge`)                                                | **Full** — native platform signals (IP, ASN, geo, TLS, HTTP version, device class, header structure)                             | CloudFront: **JA3 + JA4** (2.2.0+, via the template's origin request policy). Cloudflare Workers: Bot Management zones only. Vercel: JA4 with BotID. Fastly: none               |
+| **SDK behind a CDN you control** (your own Cloudflare / Fastly / Vercel / CloudFront distribution) | **Derived** — geo/ASN/TLS promoted from the CDN's injected headers (`deriveConnectionFromHeaders`)                               | Cloudflare: JA3 + JA4 (`cf-ja3-hash` / `cf-ja4`, plan-dependent). CloudFront: JA3 + JA4 — add the `CloudFront-Viewer-JA3/JA4-Fingerprint` headers to your origin request policy |
+| **SDK behind Akamai or Imperva**                                                                   | **Derived** — IP from `True-Client-IP` / `Incap-Client-IP`; geo/ASN from Akamai's `X-Akamai-Edgescape` when the property adds it | None                                                                                                                                                                            |
+| **SDK behind a platform-managed CDN** (Railway, Render, Fly, Heroku, …)                            | **IP only** — platform CDNs forward `x-forwarded-for` but NOT their provider-specific geo/TLS headers to customer origins        | None — no fingerprint header reaches your origin                                                                                                                                |
+| **SDK, no CDN**                                                                                    | Absent (IP from `x-forwarded-for` when a proxy sets it)                                                                          | None                                                                                                                                                                            |
 
 > "Behind Fastly" ≠ "receives Fastly's headers": platform-managed CDNs are operated by the platform, and they do not forward `Fastly-Client-IP`-style headers to your origin. If connection-layer signals or fingerprints matter to you on a managed platform, deploy the edge adapter in front (bucket 1) — that is the whole point of the TAG.
 
+> **Blocked at the edge.** Every row above sees only traffic the CDN's own
+> firewall and bot rules let through: those rules run first. Requests they
+> block reach no AstraSync code. Connect the CDN or WAF's security logs (an
+> edge-log connector on the endpoint page) to see them in Visit Intelligence
+> as "Blocked at your edge", classified from the logs and never
+> cryptographically verified.
+>
+> Edge beacons also carry the gateway's own verdict (`edgeDecision`,
+> `edgeMode`; 5.22.0), so an enforce-mode deny at the edge is recorded as a
+> deny.
+
 > **Note:** `runCommercePipeline` (on the `./transport` subpath) is the **server-side** orchestrator of commerce-protocol verification — it runs inside the AstraSync verify-access service. The edge adapters forward raw commerce artifacts to verify-access and never bundle or call the pipeline themselves.
 
 ### Which `EdgeConfig` fields each integration consumes
@@ -1117,7 +1129,7 @@ The ONE sanitiser deciding what is safe to store from an inbound agent's request
 - `sanitizeHeaders(raw)` → `{ headers, apiKeyFormat?, platformHeaders? }` — drops genuine secrets (cookies, signatures, AstraSync credential headers), reduces credential headers to a safe **format prefix** (e.g. `sk-ant-api03` — a platform signal, never the key), keeps everything else verbatim, and enforces size caps (`MAX_HEADERS`, `MAX_HEADER_VALUE_BYTES`, `MAX_TOTAL_BYTES`).
 - `extractApiKeyFormat(value)` — the safe key-format prefix of a credential header value, or `undefined` for unrecognised/opaque tokens.
 - `extractPlatformHeaders(sanitized)` — the platform-signal subset (`openai-organization`, `anthropic-version`, `x-goog-user-project`, …) of an already-sanitised map.
-- `deriveConnectionFromHeaders(headers)` — promotes CDN-injected headers (Cloudflare / Fastly / Vercel / CloudFront geo, client-IP, TLS-fingerprint, and header-structure headers — incl. `cf-ja3-hash` / `cf-ja4` and `cloudfront-viewer-ja3-fingerprint` / `-ja4-fingerprint` from 4.3.0) into the `connection` block, so SDK-only deployments behind a CDN **you control** still get connection-layer attribution (see the capture matrix for the platform-managed-CDN caveat). Values are truncated to `MAX_CONNECTION_VALUE_CHARS` (256 — the platform bound). Native edge signals remain authoritative; this is the SDK-level fallback.
+- `deriveConnectionFromHeaders(headers)` — promotes CDN-injected headers (Cloudflare / Fastly / Vercel / CloudFront / Akamai EdgeScape geo, client-IP incl. Imperva `incap-client-ip` (5.22.0), TLS-fingerprint, and header-structure headers — incl. `cf-ja3-hash` / `cf-ja4` and `cloudfront-viewer-ja3-fingerprint` / `-ja4-fingerprint` from 4.3.0) into the `connection` block, so SDK-only deployments behind a CDN **you control** still get connection-layer attribution (see the capture matrix for the platform-managed-CDN caveat). Values are truncated to `MAX_CONNECTION_VALUE_CHARS` (256 — the platform bound). Native edge signals remain authoritative; this is the SDK-level fallback.
 - `buildSdkObservedMetadata(rawHeaders)` — the one builder the express / nextjs / mcp adapters share: sanitised headers + `apiKeyFormat` + `platformHeaders` + derived `connection` + `schemaVersion`.
 - `CAPTURE_SCHEMA_VERSION` — the capture-semantics version stamped on every `ObservedMetadata` (currently `1`). Capture semantics are **frozen within a major version**: `schemaVersion` bumps only on a semantic change to what is kept/dropped/derived, never for additive signals. The verify body's `sdkVersion` disambiguates builds.
 - Types: `ObservedMetadata` (`{ headers, apiKeyFormat?, platformHeaders?, connection?, schemaVersion? }` — `connection` is populated natively by edge adapters and via CDN-header derivation by SDK adapters), `SanitizeHeadersResult`.

package/dist/adapters/express.js

@@ -60,7 +60,11 @@ var PLATFORM_HEADER_NAMES = /* @__PURE__ */ new Set([
   "x-goog-api-client",
   "user-agent",
   "x-agent-card-url",
-  "x-agent-provider"
+  "x-agent-provider",
+  // Round 31 — Web Bot Auth's Signature-Agent names the signing agent's
+  // directory host (a public URL, never a secret). It attributes a signed
+  // agent even when its UA is stock Chrome.
+  "signature-agent"
 ]);
 var PLATFORM_HEADER_PREFIXES = [
   "x-goog-",
@@ -177,7 +181,12 @@ function sanitizeHeaders(raw) {
   return result;
 }
 var CONNECTION_HEADER_SOURCES = [
-  ["ip", ["cf-connecting-ip", "fastly-client-ip", "true-client-ip", "x-real-ip"]],
+  // Round 31: `incap-client-ip` = Imperva Cloud WAF; `true-client-ip` is
+  // also Akamai's client-IP header when the property enables it.
+  [
+    "ip",
+    ["cf-connecting-ip", "fastly-client-ip", "incap-client-ip", "true-client-ip", "x-real-ip"]
+  ],
   [
     "country",
     ["cf-ipcountry", "x-vercel-ip-country", "cloudfront-viewer-country", "fastly-geo-country"]
@@ -228,6 +237,28 @@ function deriveConnectionFromHeaders(headers) {
     if (portMatch) connection.sourcePort = portMatch[1];
     if (!connection.ip && addr) connection.ip = addr.replace(/:\d+$/, "");
   }
+  {
+    const edgescape = (lower["x-akamai-edgescape"] ?? lower["akamai-edgescape"])?.trim();
+    if (edgescape) {
+      const fields = /* @__PURE__ */ new Map();
+      for (const pair of edgescape.split(",")) {
+        const i = pair.indexOf("=");
+        if (i > 0) fields.set(pair.slice(0, i).trim().toLowerCase(), pair.slice(i + 1).trim());
+      }
+      const fill = (key, field) => {
+        const v = fields.get(field);
+        if (v && !connection[key]) connection[key] = v.slice(0, MAX_CONNECTION_VALUE_CHARS);
+      };
+      fill("country", "country_code");
+      fill("region", "region_code");
+      fill("city", "city");
+      fill("latitude", "lat");
+      fill("longitude", "long");
+      fill("timeZone", "timezone");
+      fill("asn", "asnum");
+      fill("continent", "continent");
+    }
+  }
   if (!connection.ip) {
     const xff = lower["x-forwarded-for"];
     const client = xff?.split(",")[0]?.trim();
@@ -479,7 +510,7 @@ function routePolicyOf(route) {
 }
 
 // src/version.ts
-var SDK_VERSION = "5.21.1";
+var SDK_VERSION = "5.22.0";
 
 // src/http.ts
 var SDK_USER_AGENT = `astrasync-sdk/${SDK_VERSION}`;
@@ -1900,7 +1931,8 @@ var PLATFORM_AGENT_SIGNATURES = [
     uaPatterns: [/claude/i, /anthropic/i],
     agentCardPatterns: [/anthropic\.com/i, /claude\.ai/i],
     apiKeyFormatPatterns: [/^sk-ant/i],
-    headerPatterns: [/^anthropic-version/i, /^anthropic-beta/i, /^x-anthropic-/i]
+    headerPatterns: [/^anthropic-version/i, /^anthropic-beta/i, /^x-anthropic-/i],
+    wafLabelPatterns: [/\bclaude/i, /\banthropic/i]
   },
   {
     vendor: "chatgpt",
@@ -1911,7 +1943,14 @@ var PLATFORM_AGENT_SIGNATURES = [
     uaPatterns: [/chatgpt/i, /openai/i, /gpt-?\d/i, /\bGPTBot\b/i, /\bOAI-SearchBot\b/i],
     agentCardPatterns: [/openai\.com/i, /chatgpt\.com/i],
     apiKeyFormatPatterns: [/^sk-proj/i, /^sk-svcacct/i],
-    headerPatterns: [/^openai-organization/i, /^openai-project/i, /^openai-version/i]
+    headerPatterns: [
+      /^openai-organization/i,
+      /^openai-project/i,
+      /^openai-version/i,
+      // ChatGPT agent signs with Web Bot Auth under this directory host.
+      /^signature-agent: "?https:\/\/chatgpt\.com/i
+    ],
+    wafLabelPatterns: [/\bgptbot\b/i, /\bchatgpt/i, /\boai-searchbot\b/i, /\bopenai/i]
   },
   {
     vendor: "gemini",
@@ -1932,7 +1971,8 @@ var PLATFORM_AGENT_SIGNATURES = [
     ],
     agentCardPatterns: [/gemini\.google\.com/i, /ai\.google\.dev/i],
     apiKeyFormatPatterns: [/^aiza/i],
-    headerPatterns: [/^x-goog-user-project/i, /^x-goog-api-client/i]
+    headerPatterns: [/^x-goog-user-project/i, /^x-goog-api-client/i],
+    wafLabelPatterns: [/\bgoogle-extended\b/i, /\bgoogleother\b/i, /\bgemini/i]
   },
   {
     vendor: "cursor",
@@ -1951,7 +1991,31 @@ var PLATFORM_AGENT_SIGNATURES = [
     displayName: "Perplexity",
     uaPatterns: [/perplexity/i],
     agentCardPatterns: [/perplexity\.ai/i],
-    apiKeyFormatPatterns: [/^pplx-/i]
+    apiKeyFormatPatterns: [/^pplx-/i],
+    wafLabelPatterns: [/\bperplexity/i]
+  },
+  // Round 30.1: xAI's documented crawlers (GrokBot/1.0, xAI-Grok/1.0,
+  // Grok-DeepSearch/1.0). Anchored, versioned tokens — never in a stock UA.
+  {
+    vendor: "grok",
+    displayName: "xAI Grok",
+    uaPatterns: [/\bGrokBot\//i, /\bxAI-Grok\//i, /\bGrok-DeepSearch\//i],
+    agentCardPatterns: [/\bx\.ai\b/i, /grok\.com/i],
+    apiKeyFormatPatterns: [/^xai-/i],
+    wafLabelPatterns: [/\bgrok/i, /\bxai\b/i]
+  },
+  // Round 30.1: Meta AI's index and fetch crawlers. facebookexternalhit
+  // (link previews) is deliberately NOT here — it stays a generic crawler.
+  {
+    vendor: "meta-ai",
+    displayName: "Meta AI",
+    uaPatterns: [/\bMeta-WebIndexer\b/i, /\bMeta-ExternalFetcher\b/i],
+    agentCardPatterns: [/meta\.ai/i],
+    wafLabelPatterns: [
+      /\bmeta-webindexer\b/i,
+      /\bmeta-externalfetcher\b/i,
+      /\bmeta-externalagent\b/i
+    ]
   },
   // Agentic browsers — versioned-token patterns ONLY (a stock Chrome UA must
   // never match). Distinct vendor keys (not 'chatgpt'/'perplexity'): the
@@ -1999,6 +2063,7 @@ function matchPlatformSignature(signatures, input) {
   const apiKeyFormat = input.apiKeyFormat ?? "";
   const ja4 = input.ja4 ?? "";
   const asn = input.asn ?? "";
+  const wafLabels = input.wafLabels ?? [];
   const headerLines = [];
   if (input.platformHeaders) {
     for (const [name, value] of Object.entries(input.platformHeaders)) {
@@ -2012,7 +2077,8 @@ function matchPlatformSignature(signatures, input) {
     const headerMatch = headerLines.length > 0 && sig.headerPatterns?.some((p) => headerLines.some((line) => p.test(line)));
     const ja4Match = ja4.length > 0 && sig.ja4Patterns?.some((p) => p.test(ja4));
     const asnMatch = asn.length > 0 && sig.asnList?.includes(asn);
-    if (uaMatch || cardMatch || keyMatch || headerMatch || ja4Match || asnMatch) {
+    const wafMatch = wafLabels.length > 0 && sig.wafLabelPatterns?.some((p) => wafLabels.some((label) => p.test(label)));
+    if (uaMatch || cardMatch || keyMatch || headerMatch || ja4Match || asnMatch || wafMatch) {
       const uaBase = ua.split(/[\s/]/)[0]?.toLowerCase() ?? "";
       const cardHost = (() => {
         try {
@@ -2021,7 +2087,7 @@ function matchPlatformSignature(signatures, input) {
           return "";
         }
       })();
-      const keyBasis = uaBase || cardHost || apiKeyFormat || ja4 || asn && `asn${asn}` || "unknown";
+      const keyBasis = uaBase || cardHost || apiKeyFormat || ja4 || asn && `asn${asn}` || wafMatch && `waf:${wafLabels[0]?.toLowerCase()}` || "unknown";
       return {
         vendor: sig.vendor,
         fingerprintKey: `${sig.vendor}:${keyBasis}`,