@katrinalaszlo/track 0.7.1 → 0.7.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -29,7 +29,7 @@ const { agentTraffic } = require("@katrinalaszlo/track"); // or: import { agentT
29
29
  app.use(agentTraffic({ accountIdOf: (req, res) => req.user?.id }));
30
30
  ```
31
31
 
32
- Register it right after creating the app, before auth and routes. `accountIdOf` returns your customer's id (hashed before it leaves your server), or undefined. Optional `errorOf(req, res)` returns error text for failed responses (400 and above, up to 200 characters), so two errors with the same status show separately. It needs one variable, `AGENT_TRAFFIC_KEY`, in the environment or `.env`: the key belongs to one project, and events go there. No account yet? This creates a free workspace and appends its key to `.env` without printing it (the first line adds a final newline to a `.env` that lacks one):
32
+ Register it right after creating the app, before auth and routes. `accountIdOf` returns your customer's id (hashed before it leaves your server), or undefined. Optional `errorOf(req, res)` returns error text for failed responses (400 and above, up to 200 characters), so two errors with the same status show separately. It needs one variable, `AGENT_TRAFFIC_KEY`, in the environment or `.env`: the key belongs to one project, and events go there. The `.env` is read from the directory the server starts in, else from the app root: the folder whose `node_modules` holds the package (for a copy of the file inside your app, the nearest folder with a `package.json`). Since 0.7.2 no folder above those is read, so a `.env` at a monorepo root is found only when the server starts there or the package is installed in the root `node_modules`; otherwise set the variables in the environment. No account yet? This creates a free workspace and appends its key to `.env` without printing it (the first line adds a final newline to a `.env` that lacks one):
33
33
 
34
34
  ```sh
35
35
  if [ -s .env ] && [ -n "$(tail -c 1 .env)" ]; then echo >> .env; fi
@@ -40,7 +40,7 @@ Already have an account? Create a key on the Install page of your dashboard, or
40
40
 
41
41
  `AGENT_TRAFFIC_ACCOUNT_SALT` is needed when you set `accountIdOf` or call `recordOutcome`. `AGENT_TRAFFIC_PROJECT_ID` is optional: set, it is sent with each batch as before.
42
42
 
43
- Per request it sends the route pattern, the path (see below), method, status code, the `User-Agent`, `Signature-Agent` and `Accept-Language` headers, the time, the hashed customer id, the journey key if the request carried one (see below), and the text `errorOf` returns if you set it. When an agent signed its request (Web Bot Auth, `Signature` and `Signature-Input` headers present) it also sends those two headers, the request's full URL, and the other headers the signature covers, so the signature can be checked; never `Authorization` or `Cookie`, whatever the signer covered. Otherwise no bodies, no other headers, no IPs. To see which agents lead to signups and payments, call `traffic.recordOutcome({ accountId, name: "paid", value: 1999, currency: "USD", idempotencyKey: paymentEvent.id })` from your payment webhook (and `name: "signup"` from your signup handler): the outcome joins the agent session of the same customer. Events go out in batches once a second (at once on Vercel, Lambda and Netlify), and it never fails a request. The one request it holds up is a person opening a handoff link (below), for at most 2 seconds. No dependencies; TypeScript types included.
43
+ Per request it sends the route pattern, the path with secret-looking segments redacted (see below), method, status code, the `User-Agent`, `Signature-Agent` and `Accept-Language` headers, the time, the hashed customer id, the journey key if the request carried one (see below), and the text `errorOf` returns if you set it. When an agent signed its request (Web Bot Auth, `Signature` and `Signature-Input` headers present) it also sends those two headers, the request's URL (with its query only when the signature covers it), and the other headers the signature covers, so the signature can be checked; never `Authorization` or `Cookie`, whatever the signer covered. Otherwise no bodies, no other headers, no IPs. To see which agents lead to signups and payments, call `traffic.recordOutcome({ accountId, name: "paid", value: 1999, currency: "USD", idempotencyKey: paymentEvent.id })` from your payment webhook (and `name: "signup"` from your signup handler): the outcome joins the agent session of the same customer. Events go out in batches once a second (at once on Vercel, Lambda and Netlify), and it never fails a request. The one request it holds up is a person opening a handoff link (below), for at most 2 seconds. Every call to Track has a time limit: 5 seconds for a batch of events (retried like a 5xx when Track does not answer), 3 seconds for `recordOutcome` and `handoffUrl`. `recordOutcome` resolves when Track answers or the 3 seconds run out; on a timeout it logs one line and retries in the background with the same idempotency key, so an outcome Track did store is counted once (on a serverless host that retry can be lost when the function freezes). If `AGENT_TRAFFIC_ENDPOINT` is plain `http://` and not localhost, it logs one warning per process: the key and the events would travel unencrypted. No dependencies; TypeScript types included.
44
44
 
45
45
  ### Journey keys
46
46
 
@@ -52,7 +52,7 @@ To hand out keys yourself, turn on `issueJourneyKeys`:
52
52
  app.use(agentTraffic({ accountIdOf: (req, res) => req.user?.id, issueJourneyKeys: true }));
53
53
  ```
54
54
 
55
- Then every response carries the visitor's key: the one the request sent, or a new one (`j_` plus 32 hex characters) when it sent none. It goes out as the `Track-Journey` response header and as `Set-Cookie: track_j=<key>; Path=/; Max-Age=1209600; SameSite=Lax; Secure`, with `Cache-Control: private` so a shared cache never serves one visitor's key to another. It is off by default because it sets a cookie on your responses and changes their caching: that can need a line in your cookie policy or consent banner, and a CDN will stop caching those responses. Both are set before your routes run, so a route that sets its own `Cache-Control` replaces `private` (keep it private on any response that carries a key), and a raw `res.setHeader("Set-Cookie", ...)` replaces the `track_j` cookie (Express's `res.cookie` and express-session add to it instead). The request that receives a new key is sent with that key as `journey_id` and `journey_source: "issued"`, so it is the first step of the journey.
55
+ Then every response carries the visitor's key: the one the request sent, or a new one (`j_` plus 32 hex characters) when it sent none. It goes out as the `Track-Journey` response header and as `Set-Cookie: track_j=<key>; Path=/; Max-Age=1209600; SameSite=Lax; Secure; HttpOnly`, with `Cache-Control: private` so a shared cache never serves one visitor's key to another. It is off by default because it sets a cookie on your responses and changes their caching: that can need a line in your cookie policy or consent banner, and a CDN will stop caching those responses. Both are set before your routes run, so a route that sets its own `Cache-Control` replaces `private` (keep it private on any response that carries a key), and a raw `res.setHeader("Set-Cookie", ...)` replaces the `track_j` cookie (Express's `res.cookie` and express-session add to it instead). The request that receives a new key is sent with that key as `journey_id` and `journey_source: "issued"`, so it is the first step of the journey. The cookie is `HttpOnly` (since 0.7.2): only your server reads it, so scripts on your pages cannot. Browser code that needs the key can read the `Track-Journey` response header.
56
56
 
57
57
  ### Handoff links
58
58
 
@@ -70,15 +70,37 @@ app.post("/v1/checkout-links", async (req, res) => {
70
70
 
71
71
  `handoffUrl(link, { req })` asks Track for a one-time token for the journey of `req` (the agent's request, which must carry a journey key, or have one issued) and resolves with `link` plus `?track_h=<token>`. Pass `{ journeyId }` instead when you have the key without the request. It throws at once when `link` is not an absolute URL or there is no journey key. If Track cannot mint a token, it logs why and resolves with `link` unchanged, so the person still gets a working link that just does not count toward the journey.
72
72
 
73
- When the person opens the link, the middleware sees `?track_h=` and, before your routes run, sends a `handoff_opened` event carrying the token and waits up to 2 seconds for Track's answer. If Track linked it, the request's own event goes on the agent's journey (`journey_source: "issued"`), and with `issueJourneyKeys` on the response sets the person's `track_j` cookie to the agent's journey, so their sign-in and checkout land on it too. With `issueJourneyKeys` off no cookie is set: read `traffic.journeyIdOf(req)` and set the `track_j` cookie (or send the `Track-Journey` header) yourself. A token links once and for 24 hours; a used, expired or unknown token, or Track not answering in time, leaves the person on their own journey and logs one line. A token is never written to a log, and `?track_h=` never reaches `path`. Needs a Track host that takes `handoff_token` (an older one logs that it does not).
73
+ `handoffUrl` waits at most 3 seconds for Track. When the person opens the link, the middleware sees `?track_h=` and, before your routes run, sends a `handoff_opened` event carrying the token and waits up to 2 seconds for Track's answer. If Track linked it, the request's own event goes on the agent's journey (`journey_source: "issued"`), and with `issueJourneyKeys` on the response sets the person's `track_j` cookie to the agent's journey, so their sign-in and checkout land on it too. With `issueJourneyKeys` off no cookie is set: read `traffic.journeyIdOf(req)` and set the `track_j` cookie (or send the `Track-Journey` header) yourself. A token links once and for 24 hours; a used, expired or unknown token, or Track not answering in time, leaves the person on their own journey and logs one line. A token is never written to a log, and `?track_h=` never reaches `path`. Needs a Track host that takes `handoff_token` (an older one logs that it does not).
74
74
 
75
75
  Event names use the matched route's pattern and exclude router mount prefixes, which can contain raw IDs. A router mounted at `/customers/:customerId` with route `/orders/:orderId` records `orders_orderid` for every customer. Routes with the same pattern under different mounts share an event name.
76
76
 
77
77
  ### Path
78
78
 
79
- Each event also carries `path`, which Track's Paths and Attribution reports use: the path the app received, router mount prefix included, cut at the first `?` or `#`. `GET https://api.example.com/v1/users?api_key=123` is sent as `/v1/users`: never the domain, and never the query or fragment, so an API key or a `track_j` value in the query is neither sent nor logged. Segments are sent as they are, not percent-decoded, so an id in the path is sent: `/v1/users/123` arrives as `/v1/users/123`. Event names stay route patterns (`users_id`), so reports grouped by event name do not split by id; there is no option yet to leave `path` out of the event. A path that does not start with `/` or is longer than 2048 characters is left out of the event.
79
+ Each event also carries `path`, which Track's Paths and Attribution reports use: the path the app received, router mount prefix included, cut at the first `?` or `#`. `GET https://api.example.com/v1/users?api_key=123` is sent as `/v1/users`: never the domain, and never the query or fragment, so an API key or a `track_j` value in the query is neither sent nor logged. Segments are not percent-decoded. A path that does not start with `/` or is longer than 2048 characters is left out of the event.
80
80
 
81
- For a signed request (Web Bot Auth), the signature material still carries the full URL with its query, because the signature covers it and cannot be checked without it. It goes to Track ingest only, never to a log.
81
+ Since 0.7.2, path segments that look like secrets are replaced with `:token` before the event is built, so `/invite/<token>`, `/reset/<token>` and `/auth/magic/<token>` arrive as `/invite/:token`, `/reset/:token` and `/auth/magic/:token`. The same redacted path is the one written to a log. A segment counts as secret-looking when it is:
82
+
83
+ - a JWT (three dot-separated base64url parts), anywhere;
84
+ - 32 or more hex characters, anywhere;
85
+ - 20 or more base64url characters (`A-Z`, `a-z`, `0-9`, `-`, `_`) with at least one letter and one digit, that are mixed case or have no `-` or `_`, anywhere;
86
+ - right after one of `auth`, `invite(s)`, `invitation(s)`, `magic`, `magic-link`, `reset`, `reset-password`, `password-reset`, `token(s)`, `verify`, `verify-email`, `verification`, `confirm` or `activate`: also a uuid, 16 or more hex characters, or 16 or more characters of the kind above.
87
+
88
+ Ordinary ids stay as they are: `/v1/users/123` arrives as `/v1/users/123`, and so do uuids outside those routes, 24 character Mongo ObjectIds, long numeric ids and short prefixed ids such as `cus_NffrFeUfNV2Hib`. Route names stay too (`reset-password-confirmation`, `oauth2-callback`). Long random-looking ids, such as a 28 character Stripe id or a ULID, are redacted: when in doubt the rule drops a segment rather than send a secret. A token with no digit, or one shorter than the limits above, is not caught. Event names stay route patterns (`invite_token`, `users_id`), so nothing changes there.
89
+
90
+ To decide yourself, pass `pathOf(req)`. What it returns replaces the default (still cut at `?` or `#`, and left out unless it starts with `/` and fits 2048 characters); return `undefined` to leave `path` out of every event:
91
+
92
+ ```js
93
+ app.use(agentTraffic({
94
+ accountIdOf: (req, res) => req.user?.id,
95
+ pathOf: (req) => req.route?.path, // the route pattern, /invite/:token, never the value
96
+ }));
97
+ ```
98
+
99
+ That example drops the router mount prefix, because `req.baseUrl` holds the matched values of a mount such as `/teams/:teamId`, not its pattern. A request no route matched gets no path, and neither does `handoff_opened`, which is recorded before routing.
100
+
101
+ `pathOf` runs after the response, except for a request that opens a handoff link, where it runs before your routes. If it throws, that event is lost and logged (on a handoff link, `handoff_opened` goes without a path and the request carries on).
102
+
103
+ For a signed request (Web Bot Auth), the signature material carries the URL Track needs to check the signature: the raw path, and the query only when `Signature-Input` covers `@query`, `@query-param`, `@target-uri` or `@request-target`. Before 0.7.2 the query was always sent. It goes to Track ingest only, never to a log.
82
104
 
83
105
  What each event contains, and the raw HTTP API for other stacks: [alg.katrinalaszlo.com/docs](https://alg.katrinalaszlo.com/docs).
84
106
 
package/index.d.ts CHANGED
@@ -5,18 +5,24 @@ export interface AgentTrafficOptions {
5
5
  accountIdOf?: (req: any, res: any) => string | number | null | undefined;
6
6
  /** Error text for a failed response (status 400 and above), or undefined. Sent as properties.error, up to 200 characters, so two errors with the same status show separately in Fix next. */
7
7
  errorOf?: (req: any, res: any) => string | null | undefined;
8
- /** Defaults to AGENT_TRAFFIC_KEY from the environment or .env. */
8
+ /**
9
+ * The path to send for this request, instead of the default (the request path with secret-looking segments, such as
10
+ * the token in /invite/<token>, replaced by ":token"). Cut at "?" or "#"; left out unless it starts with "/" and fits
11
+ * 2048 characters. Return undefined to leave path out.
12
+ */
13
+ pathOf?: (req: any) => string | null | undefined;
14
+ /** Defaults to AGENT_TRAFFIC_KEY from the environment or .env (the working directory's, else the app root's). */
9
15
  key?: string;
10
16
  /** Defaults to AGENT_TRAFFIC_PROJECT_ID from the environment or .env. Optional: without it, events go to the key's own project. */
11
17
  projectId?: string;
12
18
  /** Defaults to AGENT_TRAFFIC_ACCOUNT_SALT from the environment or .env. Required when accountIdOf is set. */
13
19
  salt?: string;
14
- /** Defaults to AGENT_TRAFFIC_ENDPOINT, else https://alg.katrinalaszlo.com. */
20
+ /** Defaults to AGENT_TRAFFIC_ENDPOINT, else https://alg.katrinalaszlo.com. A plain http:// endpoint other than localhost logs one warning per process. */
15
21
  endpoint?: string;
16
22
  /**
17
23
  * Off by default. When true, every response carries the visitor's journey key: the one the request sent (Track-Journey
18
24
  * header, ?track_j= query parameter or track_j cookie), else a new "j_" + 32 hex key. It is set as the Track-Journey
19
- * response header and as `Set-Cookie: track_j=<key>; Path=/; Max-Age=1209600; SameSite=Lax; Secure`, with
25
+ * response header and as `Set-Cookie: track_j=<key>; Path=/; Max-Age=1209600; SameSite=Lax; Secure; HttpOnly`, with
20
26
  * `Cache-Control: private`. Opt-in because it sets a cookie on your responses and stops shared caches from storing them.
21
27
  */
22
28
  issueJourneyKeys?: boolean;
@@ -59,12 +65,15 @@ export interface OutcomeInput {
59
65
 
60
66
  export interface AgentTrafficMiddleware {
61
67
  (req: any, res: any, next: (err?: unknown) => void): void;
62
- /** Sends one outcome at once. Throws synchronously on bad input; the returned promise settles when the send finished. */
68
+ /**
69
+ * Sends one outcome at once. Throws synchronously on bad input; the returned promise settles when the send finished,
70
+ * within 3 seconds: when Track does not answer in time it logs, resolves, and retries in the background with the same key.
71
+ */
63
72
  recordOutcome(outcome: OutcomeInput): Promise<void>;
64
73
  /**
65
74
  * Resolves with `link` plus `?track_h=<token>`, a one-time token (24 hours) that puts the person who opens the link
66
75
  * on the agent's journey. Throws synchronously when link is not absolute or there is no valid journey key. When Track
67
- * cannot mint a token, logs it and resolves with `link` unchanged.
76
+ * cannot mint a token, or does not answer within 3 seconds, logs it and resolves with `link` unchanged.
68
77
  */
69
78
  handoffUrl(link: string, options: HandoffOptions): Promise<string>;
70
79
  /** The journey key the middleware recorded this request with (the agent's, when a handoff link linked it), or undefined. */
@@ -74,10 +83,15 @@ export interface AgentTrafficMiddleware {
74
83
  /**
75
84
  * Each event the middleware records carries `path`: the request path as the app received it (router mount prefix
76
85
  * included, not percent-decoded), cut at the first "?" or "#", so never the domain or the query. Left out when it does
77
- * not start with "/" or is longer than 2048 characters. Ids in path segments (/v1/users/123) are sent as they are.
86
+ * not start with "/" or is longer than 2048 characters. Ordinary ids in path segments (/v1/users/123) are sent as they
87
+ * are; segments that look like secrets (/invite/<token>, /reset/<token>, a JWT, long random strings) as ":token".
88
+ * Pass pathOf to choose the path yourself.
78
89
  * A request with a well-formed ?track_h= waits up to 2 seconds for Track to link it, before your routes run.
79
90
  */
80
91
  export function agentTraffic(options?: AgentTrafficOptions): AgentTrafficMiddleware;
81
92
 
82
93
  /** The event name for a request: its route pattern without router mount prefixes, version and "api" segments dropped, the method added for anything but GET. */
83
94
  export function eventName(req: any): string;
95
+
96
+ /** The default path redaction: each segment that looks like a secret replaced by ":token". "/invite/<token>" -> "/invite/:token". */
97
+ export function redactPath(path: string): string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@katrinalaszlo/track",
3
- "version": "0.7.1",
3
+ "version": "0.7.2",
4
4
  "description": "Express middleware that shows which AI agents call your API, how their sessions go, and which errors end them.",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -13,9 +13,11 @@
13
13
  // error text for a failed response (status 400 and above), such as an error code your handler put in res.locals, so
14
14
  // two errors with the same status show as different rows in Fix next. AGENT_TRAFFIC_KEY (required),
15
15
  // AGENT_TRAFFIC_ACCOUNT_SALT and AGENT_TRAFFIC_PROJECT_ID (optional: without it the key's own project is used) come from
16
- // the environment, or from a .env in the working directory, this file's folder or any folder above it, when the
17
- // environment lacks them; only AGENT_TRAFFIC_ lines are read, nothing else in .env is touched. issueJourneyKeys, optional and off by default, gives each visitor a journey key
18
- // on the response (a header and a track_j cookie) so its later requests join one journey.
16
+ // the environment, or from a .env in the working directory or the app root (see appRoot), when the environment lacks
17
+ // them; only AGENT_TRAFFIC_ lines are read, nothing else in .env is touched. pathOf, optional, returns the path to send
18
+ // for a request instead of the default (the request path with secret-looking segments replaced by :token).
19
+ // issueJourneyKeys, optional and off by default, gives each visitor a journey key on the response (a header and an
20
+ // HttpOnly track_j cookie) so its later requests join one journey.
19
21
  //
20
22
  // Handoffs: when an agent sends a person a link to finish (approve a payment, verify an email), add a one-time token so
21
23
  // the person's actions count toward the agent's journey:
@@ -28,21 +30,27 @@
28
30
  // and retried with the same keys.
29
31
  const { createHmac, randomBytes, randomUUID } = require("node:crypto");
30
32
  const { existsSync, readFileSync } = require("node:fs");
31
- const { dirname, join } = require("node:path");
33
+ const { dirname, join, sep } = require("node:path");
32
34
 
33
35
  // The request and response header that carries a journey key.
34
36
  const JOURNEY_HEADER = "Track-Journey";
35
37
  // The cookie name and the query parameter name.
36
38
  const JOURNEY_PARAM = "track_j";
37
39
  const JOURNEY_KEY = /^[A-Za-z0-9_-]{8,128}$/;
38
- // 14 days. Secure so the key never travels over plain http; Lax so it comes along when a person follows a link in.
39
- const JOURNEY_COOKIE_ATTRIBUTES = "Path=/; Max-Age=1209600; SameSite=Lax; Secure";
40
+ // 14 days. Secure so the key never travels over plain http; Lax so it comes along when a person follows a link in;
41
+ // HttpOnly because only the server reads it, so a script on the page cannot.
42
+ const JOURNEY_COOKIE_ATTRIBUTES = "Path=/; Max-Age=1209600; SameSite=Lax; Secure; HttpOnly";
40
43
  // The query parameter a handoff link carries, and the token shape Track mints. A token is a secret in transit: it goes
41
44
  // to Track only, never into a log or an error message.
42
45
  const HANDOFF_PARAM = "track_h";
43
46
  const HANDOFF_TOKEN = /^h_[A-Za-z0-9_-]{32,64}$/;
44
47
  // How long a request with ?track_h= waits for Track before it carries on unlinked.
45
48
  const HANDOFF_WAIT_MS = 2000;
49
+ // How long each call to Track may take before it is given up: a slow Track never holds a customer's request, payment
50
+ // webhook or serverless function for longer. A batch has the most to carry, so it gets the most time.
51
+ const SEND_TIMEOUT_MS = 5000;
52
+ const OUTCOME_TIMEOUT_MS = 3000;
53
+ const HANDOFF_URL_TIMEOUT_MS = 3000;
46
54
 
47
55
  const BATCH_MAX = 25;
48
56
  const OUTCOME_NAMES = ["signup", "key_created", "activation", "paid", "custom"];
@@ -72,14 +80,22 @@ function eventName(req) {
72
80
  return (name === "" ? "root" : name).slice(0, 64).replace(/_+$/, "");
73
81
  }
74
82
 
75
- // The .env holding AGENT_TRAFFIC_KEY: in the working directory, else in this file's folder or the nearest one above it, so
76
- // it can sit at an app's root or a monorepo's root whichever directory the server starts from.
77
- function envFile() {
78
- const candidates = [join(process.cwd(), ".env")];
83
+ // The app root: the folder whose node_modules holds this package (the first node_modules in this file's path, so a
84
+ // pnpm store or a hoisted monorepo install gives the folder that holds that node_modules). For a copy of this file
85
+ // inside an app, the nearest folder at or above it with a package.json; with none, this file's folder.
86
+ function appRoot() {
87
+ const at = __dirname.indexOf(`${sep}node_modules${sep}`);
88
+ if (at !== -1) return __dirname.slice(0, at);
79
89
  for (let dir = __dirname; ; dir = dirname(dir)) {
80
- candidates.push(join(dir, ".env"));
81
- if (dirname(dir) === dir) break;
90
+ if (existsSync(join(dir, "package.json"))) return dir;
91
+ if (dirname(dir) === dir) return __dirname;
82
92
  }
93
+ }
94
+
95
+ // The .env holding AGENT_TRAFFIC_KEY: in the working directory, else in the app root. No folder above those is read,
96
+ // so a stray .env higher up (a home folder, a parent project) is never picked up.
97
+ function envFile() {
98
+ const candidates = [join(process.cwd(), ".env"), join(appRoot(), ".env")];
83
99
  return candidates.find((file) => existsSync(file) && /^AGENT_TRAFFIC_KEY=/m.test(readFileSync(file, "utf8")));
84
100
  }
85
101
 
@@ -94,9 +110,20 @@ function agentTrafficEnv() {
94
110
  return values;
95
111
  }
96
112
 
113
+ // An endpoint on plain http sends the key and every event unencrypted. Said once per process; localhost is exempt.
114
+ let warnedPlainHttp = false;
115
+ function warnIfPlainHttp(endpoint) {
116
+ const host = /^http:\/\/(\[[^\]]*\]|[^/:?#]*)/i.exec(endpoint)?.[1]?.toLowerCase();
117
+ if (host === undefined || warnedPlainHttp) return;
118
+ if (["localhost", "127.0.0.1", "[::1]"].includes(host) || host.endsWith(".localhost")) return;
119
+ warnedPlainHttp = true;
120
+ console.error(`agent-traffic: the endpoint http://${host} is plain http, not https: the key and every event travel unencrypted. Use https outside local development.`);
121
+ }
122
+
97
123
  // Web Bot Auth: when an agent signed its request (Signature and Signature-Input headers), the event carries what ingest
98
124
  // needs to check the signature: those two headers, the method and absolute URL, and only the other headers the
99
125
  // signature covers. Never a credential header, whatever the signer covered. An unsigned request adds nothing.
126
+ // The URL keeps its query only when the signature covers it: without that, the query (which can hold an API key) stays home.
100
127
  const SIGNATURE_HEADERS = ["signature", "signature-input", "signature-agent"];
101
128
  const FORBIDDEN_COVERED = ["authorization", "proxy-authorization", "cookie", "set-cookie"];
102
129
 
@@ -109,6 +136,15 @@ function coveredHeaderNames(signatureInput) {
109
136
  return names;
110
137
  }
111
138
 
139
+ // Derived components that include the query. With any of them signed, the signature cannot be checked without it.
140
+ const QUERY_COMPONENTS = ["@query", "@query-param", "@target-uri", "@request-target"];
141
+ function coversQuery(signatureInput) {
142
+ for (const group of signatureInput.matchAll(/\(([^)]*)\)/g)) {
143
+ for (const item of group[1].matchAll(/"([^"]+)"/g)) if (QUERY_COMPONENTS.includes(item[1].toLowerCase())) return true;
144
+ }
145
+ return false;
146
+ }
147
+
112
148
  function signatureMaterial(req) {
113
149
  const signature = req.headers.signature;
114
150
  const signatureInput = req.headers["signature-input"];
@@ -120,7 +156,9 @@ function signatureMaterial(req) {
120
156
  }
121
157
  // req.protocol honours Express's trust proxy setting, so behind a proxy the URL is the one the agent signed.
122
158
  const host = req.headers.host;
123
- const url = typeof host === "string" && host !== "" ? `${req.protocol ?? "https"}://${host}${req.originalUrl ?? req.url ?? "/"}` : undefined;
159
+ const target = String(req.originalUrl ?? req.url ?? "/").split("#")[0];
160
+ const pathAndQuery = coversQuery(signatureInput) ? target : target.split("?")[0];
161
+ const url = typeof host === "string" && host !== "" ? `${req.protocol ?? "https"}://${host}${pathAndQuery}` : undefined;
124
162
  return {
125
163
  signature,
126
164
  signature_input: signatureInput,
@@ -141,6 +179,37 @@ function requestPath(req) {
141
179
  return path;
142
180
  }
143
181
 
182
+ // Route words after which the next segment is a link token: /invite/<token>, /reset/<token>, /auth/magic/<token>.
183
+ const TOKEN_ROUTE = /^(auth|invite|invites|invitation|invitations|magic|magic-link|reset|reset-password|password-reset|token|tokens|verify|verify-email|verification|confirm|activate)$/i;
184
+ const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
185
+
186
+ // A segment that looks like a secret rather than a route name or an ordinary id: a JWT; 32 or more hex characters (a
187
+ // shorter hex string, such as a 24 character Mongo ObjectId, is an id); or 20 or more base64url characters with a
188
+ // letter and a digit that are mixed case or have no - or _ (so reset-password-confirmation and oauth2-callback stay).
189
+ // Right after a TOKEN_ROUTE word the bar is lower: a uuid, 16 or more hex, or 16 or more such characters. Base64
190
+ // padding (= or %3D) is ignored.
191
+ function looksLikeSecret(segment, afterTokenRoute) {
192
+ if (afterTokenRoute && UUID.test(segment)) return true;
193
+ const s = segment.replace(/(=|%3D)+$/i, "");
194
+ if (/^[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}$/.test(s)) return true;
195
+ if (/^[0-9a-f]+$/i.test(s)) return s.length >= (afterTokenRoute ? 16 : 32);
196
+ if (s.length < (afterTokenRoute ? 16 : 20) || !/^[A-Za-z0-9_-]+$/.test(s) || !/[0-9]/.test(s) || !/[A-Za-z]/.test(s)) return false;
197
+ return (/[A-Z]/.test(s) && /[a-z]/.test(s)) || !/[-_]/.test(s);
198
+ }
199
+
200
+ // The path with each secret-looking segment replaced by :token. Numbers, uuids outside a token route, short prefixed
201
+ // ids (cus_123) and route names are kept.
202
+ function redactPath(path) {
203
+ const segments = path.split("/");
204
+ return segments.map((segment, i) => (looksLikeSecret(segment, i > 0 && TOKEN_ROUTE.test(segments[i - 1])) ? ":token" : segment)).join("/");
205
+ }
206
+
207
+ // The default path: requestPath with secret-looking segments redacted.
208
+ function redactedPath(req) {
209
+ const path = requestPath(req);
210
+ return path === undefined ? undefined : redactPath(path);
211
+ }
212
+
144
213
  // A journey key ties requests from one visitor together across sessions: the journey header, else ?track_j= in the URL,
145
214
  // else the track_j cookie. The first value that fits the ingest rule wins; one that does not is ignored, never sent.
146
215
  function journeyKeyOf(req) {
@@ -186,7 +255,7 @@ function setJourneyKey(res, journeyId) {
186
255
  res.setHeader("cache-control", "private");
187
256
  }
188
257
 
189
- function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, issueJourneyKeys = false } = {}) {
258
+ function agentTraffic({ accountIdOf, errorOf, pathOf, key, projectId, salt, endpoint, issueJourneyKeys = false } = {}) {
190
259
  const env = agentTrafficEnv();
191
260
  key ??= env.AGENT_TRAFFIC_KEY;
192
261
  projectId ??= env.AGENT_TRAFFIC_PROJECT_ID;
@@ -194,6 +263,7 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
194
263
  endpoint ??= env.AGENT_TRAFFIC_ENDPOINT ?? "https://alg.katrinalaszlo.com";
195
264
  if (!key) throw new Error("agentTraffic needs AGENT_TRAFFIC_KEY, in the environment or in .env. Run `npx agent-led-growth connect` to get one.");
196
265
  if (accountIdOf && !salt) throw new Error("agentTraffic needs AGENT_TRAFFIC_ACCOUNT_SALT, in the environment or in .env, to hash what accountIdOf returns.");
266
+ warnIfPlainHttp(endpoint);
197
267
  const projectQuery = projectId ? `?project_id=${encodeURIComponent(projectId)}` : "";
198
268
  const url = `${endpoint}/v1/events${projectQuery}`;
199
269
  const handoffsUrl = `${endpoint}/v1/handoffs${projectQuery}`;
@@ -203,13 +273,16 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
203
273
  let timer = null;
204
274
  const warned = new Set();
205
275
 
206
- async function send(batch, attempt) {
276
+ // A batch Track does not answer within timeoutMs counts as failed and is retried like a 5xx, with the same keys, so
277
+ // a batch Track did store is counted once.
278
+ async function send(batch, attempt, timeoutMs = SEND_TIMEOUT_MS) {
207
279
  let failure = null;
208
280
  try {
209
281
  const res = await fetch(url, {
210
282
  method: "POST",
211
283
  headers: { authorization: `Bearer ${key}`, "content-type": "application/json" },
212
284
  body: JSON.stringify(batch),
285
+ signal: AbortSignal.timeout(timeoutMs),
213
286
  });
214
287
  const body = await res.json();
215
288
  if (res.status === 202) {
@@ -241,14 +314,14 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
241
314
  return;
242
315
  }
243
316
  } catch (err) {
244
- failure = err.message;
317
+ failure = err.name === "TimeoutError" ? `no answer from Track within ${timeoutMs / 1000} seconds` : err.message;
245
318
  }
246
319
  if (attempt >= RETRIES_MAX) {
247
320
  console.error(`agent-traffic: batch of ${batch.length} dropped after ${attempt} attempts: ${failure}`);
248
321
  return;
249
322
  }
250
323
  console.error(`agent-traffic: batch of ${batch.length} failed (${failure}), retrying`);
251
- setTimeout(() => send(batch, attempt + 1), FLUSH_MS * 2 ** attempt).unref();
324
+ setTimeout(() => send(batch, attempt + 1, timeoutMs), FLUSH_MS * 2 ** attempt).unref();
252
325
  }
253
326
 
254
327
  function flush() {
@@ -259,7 +332,8 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
259
332
  const hashAccountId = (accountId) => createHmac("sha256", salt).update(String(accountId)).digest("hex");
260
333
 
261
334
  // An outcome is sent at once, not batched: there are few of them and each one is money or a signup. Returns a
262
- // promise that settles when the send has finished (sent, or logged and dropped); nothing it does can throw into
335
+ // promise that settles when the send has finished (sent, or logged and dropped), within 3 seconds: when Track does
336
+ // not answer in time it is logged and retried in the background with the same key. Nothing it does can throw into
263
337
  // your handler after validation. Bad input throws synchronously, at your call site, with the field named: that is
264
338
  // your bug to see now, not a rejection to find in a log later.
265
339
  function recordOutcome({ accountId, name, value, currency, idempotencyKey, occurredAt } = {}) {
@@ -283,7 +357,7 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
283
357
  properties: {},
284
358
  };
285
359
  const vercel = globalThis[VERCEL_CONTEXT]?.get?.();
286
- const sending = send([event], 1);
360
+ const sending = send([event], 1, OUTCOME_TIMEOUT_MS);
287
361
  vercel?.waitUntil?.(sending);
288
362
  return sending;
289
363
  }
@@ -294,8 +368,9 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
294
368
  }
295
369
 
296
370
  // Resolves with link plus ?track_h=<token> for the agent's journey: journeyId, else the journey of req. Bad input
297
- // throws synchronously. When Track cannot mint a token the failure is logged and the promise resolves with link
298
- // unchanged, so the person still gets a working link; it just will not count toward the agent's journey.
371
+ // throws synchronously. When Track cannot mint a token, or does not answer within 3 seconds, the failure is logged
372
+ // and the promise resolves with link unchanged, so the person still gets a working link; it just will not count
373
+ // toward the agent's journey.
299
374
  function handoffUrl(link, { req, journeyId } = {}) {
300
375
  let target;
301
376
  try {
@@ -311,6 +386,7 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
311
386
  method: "POST",
312
387
  headers: { authorization: `Bearer ${key}`, "content-type": "application/json" },
313
388
  body: JSON.stringify({ journey_id: journey }),
389
+ signal: AbortSignal.timeout(HANDOFF_URL_TIMEOUT_MS),
314
390
  });
315
391
  const body = await res.json();
316
392
  if (res.status !== 201) {
@@ -320,7 +396,8 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
320
396
  target.searchParams.set(HANDOFF_PARAM, body.token);
321
397
  return target.toString();
322
398
  } catch (err) {
323
- console.error(`agent-traffic: handoff link sent without a token: ${err.message}`);
399
+ const reason = err.name === "TimeoutError" ? `no answer from Track within ${HANDOFF_URL_TIMEOUT_MS / 1000} seconds` : err.message;
400
+ console.error(`agent-traffic: handoff link sent without a token: ${reason}`);
324
401
  return link;
325
402
  }
326
403
  })();
@@ -379,8 +456,9 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
379
456
  try {
380
457
  event = eventOf(req, res, occurredAt, journey);
381
458
  } catch (err) {
382
- // The path only: the query string can hold an API key, a journey key or a handoff token, which never go in a log.
383
- console.error(`agent-traffic: event not recorded for ${req.method} ${requestPath(req) ?? "(no path)"}: ${err.stack}`);
459
+ // The redacted path only: the query string can hold an API key, a journey key or a handoff token, and a path
460
+ // segment an invite or reset token, which never go in a log.
461
+ console.error(`agent-traffic: event not recorded for ${req.method} ${redactedPath(req) ?? "(no path)"}: ${err.stack}`);
384
462
  return;
385
463
  }
386
464
  if (SERVERLESS || vercel?.waitUntil) {
@@ -400,6 +478,17 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
400
478
  agentTrafficMiddleware.journeyIdOf = journeyIdOf;
401
479
  return agentTrafficMiddleware;
402
480
 
481
+ // The path to send: what pathOf returns, else the redacted request path. Either way cut at ? or #, and left out
482
+ // when it does not start with / or is longer than 2048 characters.
483
+ function pathFor(req) {
484
+ if (!pathOf) return redactedPath(req);
485
+ const custom = pathOf(req);
486
+ if (typeof custom !== "string") return undefined;
487
+ const path = custom.split(/[?#]/)[0];
488
+ if (!path.startsWith("/") || path.length > PATH_MAX) return undefined;
489
+ return path;
490
+ }
491
+
403
492
  function eventOf(req, res, occurredAt, journey) {
404
493
  const accountId = accountIdOf ? accountIdOf(req, res) : undefined;
405
494
  // Only for failed responses: the text that tells one 402 from another in Fix next. Capped so a long message cannot
@@ -408,7 +497,7 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
408
497
  return {
409
498
  idempotency_key: randomUUID(),
410
499
  name: eventName(req),
411
- path: requestPath(req),
500
+ path: pathFor(req),
412
501
  occurred_at: occurredAt,
413
502
  source: "sdk",
414
503
  account_id: accountId ? hashAccountId(accountId) : undefined,
@@ -423,11 +512,18 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
423
512
 
424
513
  // The event that carries a handoff token, sent before the response, so it has no status code and no account yet:
425
514
  // the person's sign-in comes on their later requests.
515
+ // It runs before your routes, so a pathOf that throws must not fail the request: the event goes without a path.
426
516
  function handoffEventOf(req, occurredAt, journey, handoffToken) {
517
+ let path;
518
+ try {
519
+ path = pathFor(req);
520
+ } catch (err) {
521
+ console.error(`agent-traffic: pathOf threw, handoff_opened sent without a path: ${err.message}`);
522
+ }
427
523
  return {
428
524
  idempotency_key: randomUUID(),
429
525
  name: "handoff_opened",
430
- path: requestPath(req),
526
+ path,
431
527
  occurred_at: occurredAt,
432
528
  source: "sdk",
433
529
  user_agent: req.headers["user-agent"],
@@ -440,4 +536,4 @@ function agentTraffic({ accountIdOf, errorOf, key, projectId, salt, endpoint, is
440
536
  }
441
537
  }
442
538
 
443
- module.exports = { eventName, agentTraffic, signatureMaterial, JOURNEY_HEADER, HANDOFF_PARAM };
539
+ module.exports = { eventName, agentTraffic, signatureMaterial, redactPath, JOURNEY_HEADER, HANDOFF_PARAM };