@scoutline-ai/middleware 0.1.0 → 0.3.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Scoutline
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -20,7 +20,7 @@ export default withScoutline(yourProxy); // or withScoutline() if you have none
 export const config = {
   matcher: [
     {
-      source: "/((?!_next/static|_next/image).*)",
+      source: "/((?!_next/static|_next/image|favicon.ico).*)",
       missing: [
         { type: "header", key: "next-router-prefetch" },
         { type: "header", key: "purpose", value: "prefetch" },
@@ -32,9 +32,24 @@ export const config = {
 
 Next.js removes prefetch headers before your proxy runs, so only the matcher can exclude them. The `missing` rule skips your whole proxy for prefetches, so leave it out if your proxy enforces auth. Prefetches then count as `unknown` traffic. They only come from browsers, so crawler and agent counts are unaffected.
 
+Your local and preview deployments can use the same key. Reports only count traffic on your site's domain and its subdomains.
+
 A proxy runs before the page renders, so normal page loads have no status or duration. Scoutline leaves those empty and only records a status when the proxy itself responds, such as a redirect.
 
-## Cloudflare Workers and Hono
+## Cloudflare Workers
+
+Pass the key in. Depending on its compatibility date and flags, a Worker may have no `process.env` to read it from.
+
+```ts
+import { env } from "cloudflare:workers";
+import { withScoutline } from "@scoutline-ai/middleware/fetch";
+
+export default { fetch: withScoutline(app.fetch, { key: env.SCOUTLINE_KEY }) };
+```
+
+Use `.dev.vars` locally and `npx wrangler secret put SCOUTLINE_KEY` in production. This sees the full response, so it records status, duration and size.
+
+## Hono and other fetch handlers
 
 ```ts
 import { withScoutline } from "@scoutline-ai/middleware/fetch";
@@ -42,7 +57,7 @@ import { withScoutline } from "@scoutline-ai/middleware/fetch";
 export default { fetch: withScoutline(app.fetch) };
 ```
 
-This sees the full response, so it records status, duration and size.
+This also records status, duration and size. If Hono runs on Workers, pass the key as shown above.
 
 ## Express and Fastify
 
@@ -74,13 +89,68 @@ It stores nothing and sets no cookies. A page links to its server request throug
 
 Browser events can be forged, so they are never `verified` and can't confirm an outcome. Reports keep them out of request counts.
 
+## Outcomes
+
+Outcomes are what your server confirms really happened, such as an order or a signup. They are never inferred from a 200.
+
+Create an outcome key on your Install page. It is separate from your middleware key.
+
+```sh
+SCOUTLINE_OUTCOME_KEY=sl_live_...
+```
+
+```ts
+import { createOutcomes } from "@scoutline-ai/middleware/outcomes";
+
+const outcomes = createOutcomes();
+
+// Runs your action, and confirms the outcome only if it succeeds.
+const order = await outcomes.wrap("order.placed", () => placeOrder(cart), (order) => ({
+  id: order.id,
+  taskId,
+  headers: request.headers,
+}));
+
+// Or confirm it yourself.
+await outcomes.confirm("signup.completed", { id: user.id, taskId });
+
+// Record a step on the way, or where it failed.
+await outcomes.record("checkout_attempted", { taskId, errorCategory: "card_declined" });
+```
+
+- **`id`** makes repeats safe: the same id counts once. It is hashed before sending. Without it, every call counts.
+- **`taskId`** is your id for one piece of work, and puts the event on that Workflow timeline.
+- **`headers`** are the incoming request's headers, which record who acted and the trace id.
+
+Nothing here throws, so a Scoutline problem can't fail your action. Each call returns `{ ok: true }` or `{ ok: false, reason }`. Events are sent straight away, at most twice for up to 2 seconds each, so don't await them if your action can't spare that.
+
+## Linking requests to a task
+
+Add your own ids to requests with `context`. Only what you return here is sent.
+
+```ts
+withScoutline(yourProxy, {
+  context: (request) => ({
+    taskId: request.headers.get("x-your-task-id"),
+    resourceId: productIdFrom(request),
+    contentVersion: process.env.VERCEL_GIT_COMMIT_SHA,
+  }),
+});
+```
+
+Requests that share a W3C `traceparent` with an event in the task also appear on its timeline, labelled as linked by trace.
+
 ## What is sent
 
-From the server: the method, pathname, classification from the user agent, and request and trace ids. From the browser: the pathname, event type, element type such as `button`, status, and load time.
+From the server: the method, pathname, user agent, whether a few standard headers were present, and request and trace ids. Images, video, fonts and similar files are recorded for crawlers and agents, and skipped for everyone else, so one page load counts once. From the browser: the pathname, event type, element type such as `button`, status, and load time.
 
 Query strings, cookies, auth headers, bodies, page text and field values are never sent.
 
-A matching user agent is recorded as `declared`, because any client can send one. Other automation is `inferred`, and everything else is `unknown`. This package never reports `verified`.
+**Device details** come from the browser's Client Hints and, where your page runs the script, its major OS version, device model, whether it has a touch screen, and a screen size group such as `small`. Exact screen sizes and full versions are not sent, and these details are never combined into an identifier. Safari and Firefox share less, so some fields stay empty rather than guessed.
+
+**The visitor's IP address** is sent to the collector, which uses it to find the country and network, then discards it. It is never stored. Country is where the network is registered, so cloud traffic may not show where the machine really is. Turn this off with `sendClientIp: false`, and country and network stay empty.
+
+The collector classifies each request, so new crawlers are recognised without updating this package. A matching user agent is recorded as `declared`, because any client can send one. Other automation is `inferred`, and everything else is `unknown`. A browser claim that doesn't add up is only `inferred` when two different kinds of evidence agree, such as missing browser headers on a hosting network. This package never reports `verified`.
 
 ## Options
 
@@ -90,3 +160,5 @@ A matching user agent is recorded as `declared`, because any client can send one
 | `endpoint` | `SCOUTLINE_ENDPOINT`, then `https://ingest.scoutline.ai` | Collector URL |
 | `sampleUnknown` | `1` | Share of unclassified traffic to send, from 0 to 1 |
 | `ignore` | none | `(pathname) => boolean` to skip a path |
+| `sendClientIp` | `true` | Send the visitor's IP for the country and network lookup. It is never stored |
+| `context` | none | `(request) => ({ taskId, sessionId, resourceId, contentVersion })` adds your own ids |

package/dist/browser-script.d.ts

@@ -1,7 +1,8 @@
 /**
  * Served from the customer's own domain at /_scoutline/s. It stays plain ES5 and
  * dependency free so it can load on any page. It collects no text, field values
- * or query strings, and stores nothing, so no cookie or consent is involved.
- * Everything it sends is an unverified claim and is revalidated in `browser.ts`.
+ * or query strings, and stores nothing. Device details stay coarse, such as a
+ * screen size group, and are never combined into an identifier. Everything it
+ * sends is an unverified claim and is revalidated in `browser.ts`.
  */
-export declare const BROWSER_SCRIPT = "(function(){\n  var ENDPOINT=\"/_scoutline/e\",MAX=20,queue=[],path=location.pathname;\n  var nav=performance.getEntriesByType?performance.getEntriesByType(\"navigation\")[0]:null;\n  var automated=navigator.webdriver===true;\n  function serverId(){\n    var timing=nav&&nav.serverTiming;\n    for(var i=0;timing&&i<timing.length;i++){\n      if(timing[i].name===\"scoutline\")return timing[i].description;\n    }\n  }\n  function send(){\n    if(!queue.length)return;\n    var now=Date.now(),events=[];\n    for(var i=0;i<queue.length;i++){\n      var e=queue[i];e.ageMs=now-e.at;delete e.at;events.push(e);\n    }\n    queue=[];\n    var body=JSON.stringify({automated:automated,events:events});\n    if(!navigator.sendBeacon||!navigator.sendBeacon(ENDPOINT,body)){\n      fetch(ENDPOINT,{method:\"POST\",body:body,keepalive:true})[\"catch\"](function(){});\n    }\n  }\n  function add(type,extra){\n    var e=extra||{};e.type=type;e.path=location.pathname;e.at=Date.now();\n    queue.push(e);\n    if(queue.length>=MAX)send();\n  }\n  function firstView(){\n    add(\"page_view\",{\n      parent:serverId(),\n      status:nav&&nav.responseStatus?nav.responseStatus:undefined,\n      durationMs:nav&&nav.duration?Math.round(nav.duration):undefined\n    });\n  }\n  function changed(){\n    if(location.pathname===path)return;\n    path=location.pathname;add(\"page_view\");\n  }\n  [\"pushState\",\"replaceState\"].forEach(function(name){\n    var original=history[name];\n    history[name]=function(){var r=original.apply(this,arguments);changed();return r;};\n  });\n  addEventListener(\"popstate\",changed);\n  document.addEventListener(\"click\",function(event){\n    var t=event.target&&event.target.closest&&event.target.closest(\"a,button,[role=button],input[type=submit]\");\n    if(t)add(\"click\",{element:t.tagName.toLowerCase()});\n  },true);\n  document.addEventListener(\"submit\",function(){add(\"form_submit\");},true);\n  addEventListener(\"error\",function(event){if(event.message)add(\"error\");});\n  addEventListener(\"unhandledrejection\",function(){add(\"error\");});\n  addEventListener(\"pagehide\",send);\n  document.addEventListener(\"visibilitychange\",function(){\n    if(document.visibilityState===\"hidden\")send();\n  });\n  if(document.readyState===\"complete\")setTimeout(firstView);\n  else addEventListener(\"load\",function(){setTimeout(firstView);});\n})();";
+export declare const BROWSER_SCRIPT = "(function(){\n  var ENDPOINT=\"/_scoutline/e\",MAX=20,queue=[],path=location.pathname;\n  var nav=performance.getEntriesByType?performance.getEntriesByType(\"navigation\")[0]:null;\n  var automated=navigator.webdriver===true;\n  var device={touch:navigator.maxTouchPoints>0};\n  var shortSide=window.screen?Math.min(screen.width,screen.height):0;\n  if(shortSide)device.screen=shortSide<600?\"small\":shortSide<900?\"medium\":\"large\";\n  if(window.matchMedia)device.coarsePointer=matchMedia(\"(pointer:coarse)\").matches;\n  var agent=navigator.userAgentData;\n  if(agent&&agent.getHighEntropyValues){\n    agent.getHighEntropyValues([\"platformVersion\",\"model\"]).then(function(values){\n      device.platform=agent.platform;device.mobile=agent.mobile;\n      device.platformVersion=values.platformVersion;device.model=values.model;\n    })[\"catch\"](function(){});\n  }\n  function serverId(){\n    var timing=nav&&nav.serverTiming;\n    for(var i=0;timing&&i<timing.length;i++){\n      if(timing[i].name===\"scoutline\")return timing[i].description;\n    }\n  }\n  function send(){\n    if(!queue.length)return;\n    var now=Date.now(),events=[];\n    for(var i=0;i<queue.length;i++){\n      var e=queue[i];e.ageMs=now-e.at;delete e.at;events.push(e);\n    }\n    queue=[];\n    var body=JSON.stringify({automated:automated,device:device,events:events});\n    if(!navigator.sendBeacon||!navigator.sendBeacon(ENDPOINT,body)){\n      fetch(ENDPOINT,{method:\"POST\",body:body,keepalive:true})[\"catch\"](function(){});\n    }\n  }\n  function add(type,extra){\n    var e=extra||{};e.type=type;e.path=location.pathname;e.at=Date.now();\n    queue.push(e);\n    if(queue.length>=MAX)send();\n  }\n  function firstView(){\n    add(\"page_view\",{\n      parent:serverId(),\n      status:nav&&nav.responseStatus?nav.responseStatus:undefined,\n      durationMs:nav&&nav.duration?Math.round(nav.duration):undefined\n    });\n  }\n  function changed(){\n    if(location.pathname===path)return;\n    path=location.pathname;add(\"page_view\");\n  }\n  [\"pushState\",\"replaceState\"].forEach(function(name){\n    var original=history[name];\n    history[name]=function(){var r=original.apply(this,arguments);changed();return r;};\n  });\n  addEventListener(\"popstate\",changed);\n  document.addEventListener(\"click\",function(event){\n    var t=event.target&&event.target.closest&&event.target.closest(\"a,button,[role=button],input[type=submit]\");\n    if(t)add(\"click\",{element:t.tagName.toLowerCase()});\n  },true);\n  document.addEventListener(\"submit\",function(){add(\"form_submit\");},true);\n  addEventListener(\"error\",function(event){if(event.message)add(\"error\");});\n  addEventListener(\"unhandledrejection\",function(){add(\"error\");});\n  addEventListener(\"pagehide\",send);\n  document.addEventListener(\"visibilitychange\",function(){\n    if(document.visibilityState===\"hidden\")send();\n  });\n  if(document.readyState===\"complete\")setTimeout(firstView);\n  else addEventListener(\"load\",function(){setTimeout(firstView);});\n})();";

package/dist/browser-script.js

@@ -1,13 +1,25 @@
 /**
  * Served from the customer's own domain at /_scoutline/s. It stays plain ES5 and
  * dependency free so it can load on any page. It collects no text, field values
- * or query strings, and stores nothing, so no cookie or consent is involved.
- * Everything it sends is an unverified claim and is revalidated in `browser.ts`.
+ * or query strings, and stores nothing. Device details stay coarse, such as a
+ * screen size group, and are never combined into an identifier. Everything it
+ * sends is an unverified claim and is revalidated in `browser.ts`.
  */
 export const BROWSER_SCRIPT = `(function(){
   var ENDPOINT="/_scoutline/e",MAX=20,queue=[],path=location.pathname;
   var nav=performance.getEntriesByType?performance.getEntriesByType("navigation")[0]:null;
   var automated=navigator.webdriver===true;
+  var device={touch:navigator.maxTouchPoints>0};
+  var shortSide=window.screen?Math.min(screen.width,screen.height):0;
+  if(shortSide)device.screen=shortSide<600?"small":shortSide<900?"medium":"large";
+  if(window.matchMedia)device.coarsePointer=matchMedia("(pointer:coarse)").matches;
+  var agent=navigator.userAgentData;
+  if(agent&&agent.getHighEntropyValues){
+    agent.getHighEntropyValues(["platformVersion","model"]).then(function(values){
+      device.platform=agent.platform;device.mobile=agent.mobile;
+      device.platformVersion=values.platformVersion;device.model=values.model;
+    })["catch"](function(){});
+  }
   function serverId(){
     var timing=nav&&nav.serverTiming;
     for(var i=0;timing&&i<timing.length;i++){
@@ -21,7 +33,7 @@ export const BROWSER_SCRIPT = `(function(){
       var e=queue[i];e.ageMs=now-e.at;delete e.at;events.push(e);
     }
     queue=[];
-    var body=JSON.stringify({automated:automated,events:events});
+    var body=JSON.stringify({automated:automated,device:device,events:events});
     if(!navigator.sendBeacon||!navigator.sendBeacon(ENDPOINT,body)){
       fetch(ENDPOINT,{method:"POST",body:body,keepalive:true})["catch"](function(){});
     }

package/dist/browser.d.ts

@@ -29,5 +29,14 @@ export declare function toBrowserEvent(raw: unknown, actor: Actor, now: number,
 } | null;
 export declare function parseBrowserBody(text: string): {
     automated: boolean;
+    device: {
+        touch: boolean | undefined;
+        coarsePointer: boolean | undefined;
+        mobile: boolean | undefined;
+        screen: string | undefined;
+        platform: string | undefined;
+        platformVersion: string | undefined;
+        model: string | undefined;
+    } | undefined;
     events: unknown[];
 } | null;

package/dist/browser.js

@@ -9,6 +9,10 @@ const MAX_DURATION_MS = 60 * 60 * 1000;
 const TYPES = new Set(["page_view", "click", "form_submit", "error"]);
 const ELEMENTS = new Set(["a", "button", "input", "div", "span"]);
 const SERVER_ID = /^[0-9a-f-]{36}$/;
+const SCREENS = new Set(["small", "medium", "large"]);
+const MAX_MODEL_CHARS = 64;
+const PLATFORM = /^[A-Za-z][A-Za-z0-9 ]{0,31}$/;
+const MODEL = /^[\w .()+-]+$/;
 /**
  * Anything in the page can be forged, so only a same-origin browser request is
  * accepted. This keeps out other sites, not a determined client, which is why
@@ -82,6 +86,31 @@ export function toBrowserEvent(raw, actor, now, sampleRate) {
         classifierVersion: BROWSER_CLASSIFIER_VERSION,
     };
 }
+/** Forged values can only mislabel the page that sent them. */
+function deviceOf(raw) {
+    if (typeof raw !== "object" || raw === null)
+        return undefined;
+    const { coarsePointer, mobile, model, platform, platformVersion, screen, touch } = raw;
+    const major = typeof platformVersion === "string"
+        ? /^\d{1,4}/.exec(platformVersion)?.[0]
+        : undefined;
+    return {
+        touch: typeof touch === "boolean" ? touch : undefined,
+        coarsePointer: typeof coarsePointer === "boolean" ? coarsePointer : undefined,
+        mobile: typeof mobile === "boolean" ? mobile : undefined,
+        screen: typeof screen === "string" && SCREENS.has(screen) ? screen : undefined,
+        platform: typeof platform === "string" && PLATFORM.test(platform)
+            ? platform
+            : undefined,
+        // The major version is enough, and the rest narrows down one machine.
+        platformVersion: major,
+        model: typeof model === "string" &&
+            model.length <= MAX_MODEL_CHARS &&
+            MODEL.test(model)
+            ? model
+            : undefined,
+    };
+}
 export function parseBrowserBody(text) {
     if (text.length > MAX_BODY_CHARS)
         return null;
@@ -92,6 +121,7 @@ export function parseBrowserBody(text) {
         }
         return {
             automated: body.automated === true,
+            device: deviceOf(body.device),
             events: body.events,
         };
     }

package/dist/client-ip.d.ts

@@ -0,0 +1,31 @@
+/**
+ * Adapted from get-client-ip.ts in OpenPanel's Express SDK. The header order
+ * here puts platform-set headers first.
+ *
+ * MIT License
+ *
+ * Copyright (c) 2026 OpenPanel
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+/** Node reports an IPv4 client as `::ffff:1.2.3.4` on a dual-stack socket. */
+export declare function withoutIpv4Prefix(ip: string): string;
+export declare function usableIp(value: string | null | undefined): string | undefined;
+/** The first public address a proxy reported for the visitor. */
+export declare function clientIpFromHeaders(headers: Headers): string | undefined;

package/dist/client-ip.js

@@ -0,0 +1,83 @@
+/**
+ * Adapted from get-client-ip.ts in OpenPanel's Express SDK. The header order
+ * here puts platform-set headers first.
+ *
+ * MIT License
+ *
+ * Copyright (c) 2026 OpenPanel
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+const HEADER_ORDER = [
+    "cf-connecting-ip",
+    "x-vercel-forwarded-for",
+    "true-client-ip",
+    "x-real-ip",
+    "x-forwarded-for",
+    "fastly-client-ip",
+    "x-cluster-client-ip",
+    "x-appengine-user-ip",
+    "do-connecting-ip",
+    "x-nf-client-connection-ip",
+    "forwarded",
+];
+const IPV4 = /^(\d{1,3}\.){3}\d{1,3}$/;
+const IPV6 = /^([0-9a-fA-F]{0,4}:){2,7}[0-9a-fA-F]{0,4}$/;
+/** Node reports an IPv4 client as `::ffff:1.2.3.4` on a dual-stack socket. */
+export function withoutIpv4Prefix(ip) {
+    return ip.startsWith("::ffff:") ? ip.slice("::ffff:".length) : ip;
+}
+function isPublicIp(address) {
+    const ip = withoutIpv4Prefix(address);
+    if (ip === "::1")
+        return false;
+    if (/^(fc|fd|fe8|fe9|fea|feb)/i.test(ip) && ip.includes(":"))
+        return false;
+    if (/^(127|10)\./.test(ip))
+        return false;
+    if (ip.startsWith("192.168.") || ip.startsWith("169.254."))
+        return false;
+    if (ip.startsWith("172.")) {
+        const second = Number.parseInt(ip.split(".")[1] ?? "0", 10);
+        if (second >= 16 && second <= 31)
+            return false;
+    }
+    return true;
+}
+export function usableIp(value) {
+    const ip = value ? withoutIpv4Prefix(value.trim()) : "";
+    if (!(IPV4.test(ip) || IPV6.test(ip)))
+        return undefined;
+    return isPublicIp(ip) ? ip : undefined;
+}
+/** The first public address a proxy reported for the visitor. */
+export function clientIpFromHeaders(headers) {
+    for (const name of HEADER_ORDER) {
+        const value = headers.get(name);
+        if (!value)
+            continue;
+        const candidate = name === "forwarded"
+            ? /for="?\[?([^\]",;]+)/i.exec(value)?.[1]
+            : value.split(",")[0];
+        const ip = usableIp(candidate);
+        if (ip)
+            return ip;
+    }
+    return undefined;
+}

package/dist/config.d.ts

@@ -0,0 +1,2 @@
+export declare const DEFAULT_ENDPOINT = "https://ingest.scoutline.ai";
+export declare function env(name: string): string | undefined;

package/dist/config.js

@@ -0,0 +1,4 @@
+export const DEFAULT_ENDPOINT = "https://ingest.scoutline.ai";
+export function env(name) {
+    return typeof process === "undefined" ? undefined : process?.env?.[name];
+}

package/dist/fetch.d.ts

@@ -6,5 +6,8 @@ import { type ScoutlineOptions } from "./index.js";
  * also adds the browser script to HTML pages, so no code change is needed.
  *
  *   export default { fetch: withScoutline(app.fetch) };
+ *
+ * Workers should pass `{ key: env.SCOUTLINE_KEY }` from `cloudflare:workers`,
+ * because older compatibility dates have no `process.env` to read it from.
  */
 export declare function withScoutline<Args extends unknown[]>(handler: (request: Request, ...args: Args) => Response | Promise<Response>, options?: ScoutlineOptions): (request: Request, ...args: Args) => Promise<Response>;

package/dist/fetch.js

@@ -31,6 +31,9 @@ function contentLength(response) {
  * also adds the browser script to HTML pages, so no code change is needed.
  *
  *   export default { fetch: withScoutline(app.fetch) };
+ *
+ * Workers should pass `{ key: env.SCOUTLINE_KEY }` from `cloudflare:workers`,
+ * because older compatibility dates have no `process.env` to read it from.
  */
 export function withScoutline(handler, options) {
     const scoutline = createScoutline(options);

package/dist/index.d.ts

@@ -1,4 +1,16 @@
 export { classify } from "./classify.js";
+/**
+ * What your app knows about a request that Scoutline can't see. Everything is
+ * optional, and only what you return here leaves your server.
+ */
+export type RequestContext = {
+    /** Your id for the piece of work, shared by the requests and outcomes in it. */
+    taskId?: string | null;
+    sessionId?: string | null;
+    /** The business object, such as a product, so its URLs count as one. */
+    resourceId?: string | null;
+    contentVersion?: string | null;
+};
 export type ScoutlineOptions = {
     /** Ingest key. Falls back to the SCOUTLINE_KEY environment variable. */
     key?: string;
@@ -8,10 +20,19 @@ export type ScoutlineOptions = {
     sampleUnknown?: number;
     /** Skip a path entirely. Runs on the pathname only. */
     ignore?: (pathname: string) => boolean;
+    /** Adds your own ids to a request. A failure here only leaves them out. */
+    context?: (request: Request) => RequestContext | null | undefined;
+    /**
+     * Sends the visitor's IP so the collector can find its country and network.
+     * The collector never stores it. Default true.
+     */
+    sendClientIp?: boolean;
 };
 export type Observation = {
     /** Set by adapters that must announce the id before the response finishes. */
     eventId?: string;
+    /** Set by adapters that can see the connection, not just the headers. */
+    clientIp?: string;
     /** Null when the response isn't visible from this position in the stack. */
     status?: number | null;
     durationMs?: number | null;
@@ -30,13 +51,40 @@ export type Scoutline = {
      * Serves the browser script and its events from the site's own domain.
      * Returns null for every other path. Must run before any auth check.
      */
-    handle(request: Request): Promise<Response> | null;
+    handle(request: Request, clientIp?: string): Promise<Response> | null;
 };
 /** Only full page loads carry a server id for the browser script to read. */
 export declare function isDocumentRequest(request: Request): boolean;
 export declare function serverTiming(eventId: string): string;
-/** The site's hostname. A proxy may list several, nearest client first. */
+/** A proxy may list several hosts, nearest client first. */
+export declare function hostFromHeaders(headers: Headers): string | null;
+/** The site's hostname. */
 export declare function hostOf(request: Request): string | undefined;
+/**
+ * What the collector needs to classify a request and name its device. The
+ * three Client Hints are the browser's own low-detail summary. For the others,
+ * only whether a header exists is sent. Rules live on the collector, so they can
+ * change without a new release of this package.
+ */
+export declare function requestFacts(headers: Headers, options: {
+    sendClientIp?: boolean;
+    clientIp?: string;
+}): {
+    userAgent: string | undefined;
+    clientIp: string | undefined;
+    clientHints: {
+        brands: string | undefined;
+        mobile: string | undefined;
+        platform: string | undefined;
+    };
+    requestShape: {
+        clientHints: boolean;
+        fetchMetadata: boolean;
+        acceptLanguage: boolean;
+    };
+};
+/** The W3C trace id from `traceparent`, a claim until something validates it. */
+export declare function traceId(headers: Headers): string | undefined;
 /**
  * Failure here must never reach the customer's site. A missing key turns the
  * client into a no-op, and a full queue drops the oldest events.

package/dist/index.js

@@ -1,8 +1,9 @@
 import { EVENTS_PATH, isSameOrigin, parseBrowserBody, SCRIPT_PATH, toBrowserEvent, } from "./browser.js";
 import { BROWSER_SCRIPT } from "./browser-script.js";
 import { classify, CLASSIFIER_VERSION } from "./classify.js";
+import { clientIpFromHeaders } from "./client-ip.js";
+import { DEFAULT_ENDPOINT, env } from "./config.js";
 export { classify } from "./classify.js";
-const DEFAULT_ENDPOINT = "https://ingest.scoutline.ai";
 const MAX_QUEUED = 500;
 const MAX_BATCH = 100;
 const SEND_TIMEOUT_MS = 2000;
@@ -12,11 +13,18 @@ const REJECTED_BACKOFF_MS = 60 * 1000;
 const RATE_LIMITED_BACKOFF_MS = 60 * 1000;
 const MAX_FIELD_CHARS = 128;
 const MAX_URL_CHARS = 2048;
+const MAX_USER_AGENT_CHARS = 512;
+const MAX_HINT_CHARS = 256;
 /** Caps browser events per server instance, so a page can't flood the queue. */
 const MAX_BROWSER_EVENTS_PER_MINUTE = 2000;
 const SCRIPT_CACHE_SECONDS = 3600;
-/** Prefetches aren't visits, and assets aren't what agents come for. */
-const IGNORED_PATH = /^\/(?:_next|_vercel|\.well-known\/vercel)\/|\.(?:css|js|mjs|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf)$/i;
+const FRAMEWORK_PATH = /^\/(?:_next|_vercel|\.well-known\/vercel)\//;
+/**
+ * A person's page load fetches these as a side effect, so they would only add
+ * noise. A crawler chooses to fetch them, and they can be most of what it costs
+ * to serve, so they stay for anyone identified. PDFs always stay.
+ */
+const ASSET_PATH = /\.(?:css|js|mjs|map|png|jpe?g|gif|webp|avif|svg|ico|woff2?|ttf|otf|mp4|m4v|webm|mov|ogv|mp3|m4a|aac|flac|wav|ogg|opus)$/i;
 /** Only full page loads carry a server id for the browser script to read. */
 export function isDocumentRequest(request) {
     return (request.method === "GET" &&
@@ -41,9 +49,6 @@ function ownRoute(request) {
     const { pathname } = new URL(request.url);
     return pathname === SCRIPT_PATH || pathname === EVENTS_PATH ? pathname : null;
 }
-function env(name) {
-    return typeof process === "undefined" ? undefined : process?.env?.[name];
-}
 function limited(value, max = MAX_FIELD_CHARS) {
     return value && value.length <= max ? value : undefined;
 }
@@ -54,34 +59,71 @@ function interfaceFor(pathname, headers) {
         return "api";
     return headers.get("accept")?.includes("application/json") ? "api" : "web";
 }
-/** The site's hostname. A proxy may list several, nearest client first. */
+/** A proxy may list several hosts, nearest client first. */
+export function hostFromHeaders(headers) {
+    const forwarded = headers.get("x-forwarded-host")?.split(",")[0];
+    return forwarded?.trim() || headers.get("host") || null;
+}
+/** The site's hostname. */
 export function hostOf(request) {
-    const forwarded = request.headers.get("x-forwarded-host")?.split(",")[0];
-    const host = forwarded?.trim() ||
-        request.headers.get("host") ||
-        new URL(request.url).host;
+    const host = hostFromHeaders(request.headers) ?? new URL(request.url).host;
     return limited(host.toLowerCase(), 255);
 }
+/**
+ * What the collector needs to classify a request and name its device. The
+ * three Client Hints are the browser's own low-detail summary. For the others,
+ * only whether a header exists is sent. Rules live on the collector, so they can
+ * change without a new release of this package.
+ */
+export function requestFacts(headers, options) {
+    return {
+        userAgent: limited(headers.get("user-agent"), MAX_USER_AGENT_CHARS),
+        clientIp: options.sendClientIp === false
+            ? undefined
+            : (options.clientIp ?? clientIpFromHeaders(headers)),
+        clientHints: {
+            brands: limited(headers.get("sec-ch-ua"), MAX_HINT_CHARS),
+            mobile: limited(headers.get("sec-ch-ua-mobile"), MAX_HINT_CHARS),
+            platform: limited(headers.get("sec-ch-ua-platform"), MAX_HINT_CHARS),
+        },
+        requestShape: {
+            clientHints: headers.has("sec-ch-ua"),
+            fetchMetadata: headers.has("sec-fetch-mode") || headers.has("sec-fetch-site"),
+            acceptLanguage: headers.has("accept-language"),
+        },
+    };
+}
 function isPrefetch(headers) {
     return (headers.has("next-router-prefetch") ||
         headers.get("purpose") === "prefetch" ||
         headers.get("sec-purpose")?.includes("prefetch") === true);
 }
 /** The W3C trace id from `traceparent`, a claim until something validates it. */
-function traceId(headers) {
+export function traceId(headers) {
     return limited(headers.get("traceparent")?.split("-")[1] ?? null);
 }
-function toEvent(request, observation, sampleUnknown, ignore) {
+function contextOf(request, read) {
+    try {
+        return read?.(request) ?? {};
+    }
+    catch {
+        return {};
+    }
+}
+function toEvent(request, observation, sampleUnknown, options) {
     const { pathname } = new URL(request.url);
-    if (IGNORED_PATH.test(pathname) || ignore?.(pathname))
+    if (FRAMEWORK_PATH.test(pathname) || options.ignore?.(pathname))
         return null;
     const { headers } = request;
     if (isPrefetch(headers))
         return null;
     const actor = classify(headers.get("user-agent"));
+    if (actor.class === "unknown" && ASSET_PATH.test(pathname))
+        return null;
     const sampled = actor.class === "unknown" && sampleUnknown < 1;
     if (sampled && Math.random() >= sampleUnknown)
         return null;
+    const context = contextOf(request, options.context);
     return {
         eventId: observation.eventId ?? crypto.randomUUID(),
         occurredAt: Date.now(),
@@ -91,11 +133,19 @@ function toEvent(request, observation, sampleUnknown, ignore) {
         // The pathname only: queries, cookies, headers and bodies never leave here.
         url: pathname.slice(0, MAX_URL_CHARS),
         host: hostOf(request),
+        ...requestFacts(headers, {
+            sendClientIp: options.sendClientIp,
+            clientIp: observation.clientIp,
+        }),
         interface: interfaceFor(pathname, headers),
         requestId: limited(headers.get("x-vercel-id") ??
             headers.get("cf-ray") ??
             headers.get("x-request-id")),
         traceId: traceId(headers),
+        taskId: limited(context.taskId ?? null),
+        sessionId: limited(context.sessionId ?? null),
+        resourceId: limited(context.resourceId ?? null),
+        contentVersion: limited(context.contentVersion ?? null),
         status: observation.status ?? undefined,
         durationMs: observation.durationMs ?? undefined,
         bytes: observation.bytes ?? undefined,
@@ -212,7 +262,7 @@ export function createScoutline(options = {}) {
     }
     function observe(request, observation = {}) {
         try {
-            const event = toEvent(request, observation, sampleUnknown, options.ignore);
+            const event = toEvent(request, observation, sampleUnknown, options);
             if (!event)
                 return undefined;
             enqueue(event);
@@ -240,7 +290,7 @@ export function createScoutline(options = {}) {
         }
         return actor;
     }
-    async function acceptBrowserEvents(request) {
+    async function acceptBrowserEvents(request, clientIp) {
         try {
             if (request.method !== "POST")
                 return emptyResponse(405);
@@ -256,10 +306,21 @@ export function createScoutline(options = {}) {
             if (sampled && Math.random() >= sampleUnknown)
                 return emptyResponse(204);
             const now = Date.now();
+            const facts = requestFacts(request.headers, {
+                sendClientIp: options.sendClientIp,
+                clientIp,
+            });
             for (const raw of body.events) {
                 const event = toBrowserEvent(raw, actor, now, sampled ? sampleUnknown : 1);
-                if (event)
-                    enqueue({ ...event, host: hostOf(request) });
+                if (!event)
+                    continue;
+                const device = event.action === "browser:page_view" ? body.device : null;
+                enqueue({
+                    ...event,
+                    ...facts,
+                    host: hostOf(request),
+                    device: device ?? undefined,
+                });
             }
             return emptyResponse(204);
         }
@@ -267,11 +328,11 @@ export function createScoutline(options = {}) {
             return emptyResponse(204);
         }
     }
-    function handle(request) {
+    function handle(request, clientIp) {
         const route = ownRoute(request);
         if (route === SCRIPT_PATH)
             return Promise.resolve(scriptResponse());
-        return route ? acceptBrowserEvents(request) : null;
+        return route ? acceptBrowserEvents(request, clientIp) : null;
     }
     return { observe, flush, handle };
 }

package/dist/node.js

@@ -1,4 +1,5 @@
 import { EVENTS_PATH, SCRIPT_PATH } from "./browser.js";
+import { usableIp } from "./client-ip.js";
 import { createScoutline, isDocumentRequest, serverTiming, } from "./index.js";
 const MAX_BODY_BYTES = 32 * 1024;
 function toRequest(request, body) {
@@ -16,6 +17,13 @@ function toRequest(request, body) {
         body,
     });
 }
+/**
+ * A public peer is the visitor, and can't fake that the way it can fake a
+ * header. A private one is a proxy, so its headers are used instead.
+ */
+function connectionIp(request) {
+    return usableIp(request.socket?.remoteAddress);
+}
 /** A body parser may have run first, and a stream can only be read once. */
 async function readBody(request) {
     if (typeof request.body === "string")
@@ -34,7 +42,7 @@ async function readBody(request) {
 }
 async function serveOwnRoute(scoutline, request, response) {
     const body = request.method === "POST" ? await readBody(request) : undefined;
-    const answer = await scoutline.handle(toRequest(request, body));
+    const answer = await scoutline.handle(toRequest(request, body), connectionIp(request));
     const text = (await answer?.text()) ?? "";
     response.writeHead(answer?.status ?? 204, Object.fromEntries(answer?.headers ?? []));
     response.end(text);
@@ -63,6 +71,7 @@ function track(scoutline, request, response) {
         done = true;
         scoutline.observe(web, {
             eventId,
+            clientIp: connectionIp(request),
             status: completed ? response.statusCode : null,
             bytes: completed ? contentLength(response) : null,
             durationMs: Date.now() - started,

package/dist/outcomes.d.ts

@@ -0,0 +1,56 @@
+export type OutcomeOptions = {
+    /** Outcome key. Falls back to the SCOUTLINE_OUTCOME_KEY environment variable. */
+    key?: string;
+    /** Collector URL, for self-hosting or testing. Falls back to SCOUTLINE_ENDPOINT. */
+    endpoint?: string;
+    /**
+     * Sends the visitor's IP from `headers` so the collector can find its country
+     * and network. The collector never stores it. Default true.
+     */
+    sendClientIp?: boolean;
+};
+export type EventDetails = {
+    /**
+     * Makes repeats safe: the same id counts once, so use your order or signup
+     * id. It is hashed before sending, so the id itself never leaves your server.
+     */
+    id?: string;
+    /** Your id for the piece of work. Shows this event in its Workflow timeline. */
+    taskId?: string;
+    sessionId?: string;
+    resourceId?: string;
+    contentVersion?: string;
+    /** Your own opaque reference for who the agent acted for. Never an email. */
+    principalRef?: string;
+    accountRef?: string;
+    /** The incoming request's headers, to record who acted and the trace id. */
+    headers?: Headers | Record<string, string | string[] | undefined>;
+    errorCategory?: string;
+    status?: number;
+    durationMs?: number;
+};
+/** Something the server knows happened, short of a confirmed outcome. */
+export type ServerEventFamily = "search" | "comparison" | "tool_invocation" | "authentication_required" | "permission_denied" | "quote_created" | "checkout_attempted" | "payment_required" | "payment_settled" | "approval_requested" | "human_takeover" | "workflow_resumed";
+export type SendResult = {
+    ok: true;
+} | {
+    ok: false;
+    reason: string;
+};
+/**
+ * For the things only your server can confirm. Outcomes are never inferred from
+ * a 200, because a page loading doesn't prove the order went through.
+ *
+ * Nothing here throws, so a Scoutline problem can't fail your action. Events
+ * are sent straight away rather than queued, because a serverless function can
+ * stop as soon as it returns. Each one is sent at most twice, up to 2 seconds
+ * each. Don't await it if your action can't spare that.
+ *
+ *   const outcomes = createOutcomes();
+ *   await outcomes.confirm("order.placed", { id: order.id, taskId });
+ */
+export declare function createOutcomes(options?: OutcomeOptions): {
+    confirm: (outcomeKey: string, details?: EventDetails) => Promise<SendResult>;
+    record: (family: ServerEventFamily, details?: EventDetails) => Promise<SendResult>;
+    wrap: <Result>(outcomeKey: string, action: () => Promise<Result>, details?: EventDetails | ((result: Result) => EventDetails)) => Promise<Result>;
+};

package/dist/outcomes.js

@@ -0,0 +1,144 @@
+import { classify, CLASSIFIER_VERSION } from "./classify.js";
+import { DEFAULT_ENDPOINT, env } from "./config.js";
+import { hostFromHeaders, requestFacts, traceId } from "./index.js";
+const SEND_TIMEOUT_MS = 2000;
+const MAX_ID_CHARS = 128;
+function toHeaders(headers) {
+    if (!headers || headers instanceof Headers)
+        return headers ?? new Headers();
+    const result = new Headers();
+    for (const [name, value] of Object.entries(headers)) {
+        if (value !== undefined) {
+            result.set(name, Array.isArray(value) ? value.join(", ") : value);
+        }
+    }
+    return result;
+}
+function bounded(value) {
+    return value && value.length <= MAX_ID_CHARS ? value : undefined;
+}
+async function stableId(parts) {
+    const bytes = new TextEncoder().encode(parts.join("\0"));
+    const digest = await crypto.subtle.digest("SHA-256", bytes);
+    const hex = [...new Uint8Array(digest)]
+        .map((byte) => byte.toString(16).padStart(2, "0"))
+        .join("");
+    return `sdk_${hex.slice(0, 40)}`;
+}
+/** Retries can't help with these, so only a failed delivery goes again. */
+function shouldRetry(status) {
+    return status === undefined || status >= 500;
+}
+/**
+ * For the things only your server can confirm. Outcomes are never inferred from
+ * a 200, because a page loading doesn't prove the order went through.
+ *
+ * Nothing here throws, so a Scoutline problem can't fail your action. Events
+ * are sent straight away rather than queued, because a serverless function can
+ * stop as soon as it returns. Each one is sent at most twice, up to 2 seconds
+ * each. Don't await it if your action can't spare that.
+ *
+ *   const outcomes = createOutcomes();
+ *   await outcomes.confirm("order.placed", { id: order.id, taskId });
+ */
+export function createOutcomes(options = {}) {
+    const key = options.key ?? env("SCOUTLINE_OUTCOME_KEY");
+    const endpoint = (options.endpoint ||
+        env("SCOUTLINE_ENDPOINT") ||
+        DEFAULT_ENDPOINT).replace(/\/+$/, "");
+    if (!key) {
+        console.warn("[scoutline] Missing SCOUTLINE_OUTCOME_KEY, so outcomes aren't sent.");
+    }
+    async function post(event) {
+        let status;
+        try {
+            const response = await fetch(`${endpoint}/v1/events`, {
+                method: "POST",
+                headers: {
+                    Authorization: `Bearer ${key}`,
+                    "Content-Type": "application/json",
+                },
+                body: JSON.stringify({ events: [event] }),
+                signal: AbortSignal.timeout(SEND_TIMEOUT_MS),
+            });
+            status = response.status;
+            if (status !== 202) {
+                await response.body?.cancel();
+                return { status };
+            }
+            const body = (await response.json());
+            return { status, rejected: body.rejected?.[0]?.reason };
+        }
+        catch {
+            return { status };
+        }
+    }
+    async function send(family, outcomeKey, details) {
+        if (!key)
+            return { ok: false, reason: "no_key" };
+        try {
+            const headers = toHeaders(details.headers);
+            const userAgent = headers.get("user-agent");
+            const event = {
+                eventId: details.id
+                    ? await stableId([family, outcomeKey ?? "", details.id])
+                    : crypto.randomUUID(),
+                occurredAt: Date.now(),
+                family,
+                outcomeKey,
+                actor: classify(userAgent),
+                ...requestFacts(headers, { sendClientIp: options.sendClientIp }),
+                host: hostFromHeaders(headers)?.toLowerCase() ?? undefined,
+                interface: "app",
+                traceId: traceId(headers),
+                taskId: bounded(details.taskId),
+                sessionId: bounded(details.sessionId),
+                resourceId: bounded(details.resourceId),
+                contentVersion: bounded(details.contentVersion),
+                principalRef: bounded(details.principalRef),
+                accountRef: bounded(details.accountRef),
+                errorCategory: bounded(details.errorCategory),
+                status: details.status,
+                durationMs: details.durationMs,
+                sampleRate: 1,
+                classifierVersion: CLASSIFIER_VERSION,
+            };
+            let result = await post(event);
+            if (shouldRetry(result.status))
+                result = await post(event);
+            if (result.status === 202 && !result.rejected)
+                return { ok: true };
+            const reason = result.rejected ?? `status_${result.status ?? "none"}`;
+            console.warn(`[scoutline] ${family} wasn't accepted: ${reason}`);
+            return { ok: false, reason };
+        }
+        catch {
+            return { ok: false, reason: "failed" };
+        }
+    }
+    /** Records that the outcome really happened. */
+    function confirm(outcomeKey, details = {}) {
+        return send("outcome_completed", outcomeKey, details);
+    }
+    /** Records a server event, such as a failed checkout, on the timeline. */
+    function record(family, details = {}) {
+        return send(family, undefined, details);
+    }
+    /**
+     * Runs your action and confirms the outcome only if it succeeds. A failure is
+     * rethrown unchanged. Pass a function to take the id from the result.
+     */
+    async function wrap(outcomeKey, action, details = {}) {
+        const result = await action();
+        let resolved = {};
+        try {
+            resolved = typeof details === "function" ? details(result) : details;
+        }
+        catch {
+            // Observation is optional. The customer's action already succeeded.
+        }
+        await confirm(outcomeKey, resolved);
+        return result;
+    }
+    return { confirm, record, wrap };
+}

package/package.json

@@ -1,6 +1,7 @@
 {
   "name": "@scoutline-ai/middleware",
-  "version": "0.1.0",
+  "version": "0.3.0",
+  "license": "MIT",
   "description": "In-path agent observation for Scoutline. Fails open and adds no work to the hot path.",
   "type": "module",
   "files": [
@@ -11,6 +12,10 @@
       "types": "./dist/index.d.ts",
       "default": "./dist/index.js"
     },
+    "./outcomes": {
+      "types": "./dist/outcomes.d.ts",
+      "default": "./dist/outcomes.js"
+    },
     "./next": {
       "types": "./dist/next.d.ts",
       "default": "./dist/next.js"