midline-agent 0.3.0 → 0.4.1

package/README.md

@@ -1,6 +1,7 @@
 # midline-agent
 
-**The Node.js SDK for Midline** — request and error monitoring with security and threat detection.
+**SDK for Midline** — request and error monitoring with security and threat detection.
+It also ships **`midline-agent/browser`** for web apps: see [Browser apps](#browser-apps).
 
 Midline itself is not tied to Node. Every event lands on the same stream through a plain JSON
 endpoint, so a Django, Rails, Laravel, Spring, Go or .NET service reports exactly what an Express
@@ -102,7 +103,8 @@ bootstrap();
 ```
 
 Nest handles exceptions in its own filters, so they rarely reach Express error middleware. The
-request middleware still records every 4xx/5xx response.
+request middleware still records every 4xx/5xx response, and Midline groups the 5xx responses into
+Issues by method, route and status.
 
 ### Plain Node `http`
 
@@ -283,8 +285,9 @@ TARGET_API_URL=http://localhost:4000
 ## Console output
 
 Set `captureConsole: true` (or `MIDLINE_CAPTURE_CONSOLE=true`) and whatever the process prints
-shows up on the dashboard's Logs page as `console` events, one per line, next to your requests. That
-covers `console.log`, Nest's logger, pino, winston: anything written to stdout or stderr.
+is sent as `console` events, one per line, and shows up on the dashboard's **Terminal** page. That
+covers `console.log`, Nest's logger, pino, winston: anything written to stdout or stderr. The Logs page
+stays request traffic.
 
 ```ts
 MidlineAgent.init({ apiKey: process.env.MIDLINE_API_KEY, captureConsole: true });
@@ -299,7 +302,7 @@ const app = await NestFactory.create(AppModule); // startup lines are captured f
   medium, and everything else is low.
 - Up to 100 lines a second are sent, with room for a 1,000-line burst such as Nest mapping its routes
   at startup. Lines past that still print but aren't sent.
-- Printed lines don't count towards request totals, error rates or latency.
+- Printed lines don't count towards request totals, error rates, latency or Issues.
 - It needs a Midline server that knows the `console` event type. An older server refuses the first
   line; the agent then turns console capture off, says so once, and keeps sending requests and errors.
 - The agent's own diagnostics are never captured.
@@ -372,6 +375,88 @@ app.get("/x", (req, res) => { logger.info({ requestId: getRequestContext(req)?.r
 
 ---
 
+## Browser apps
+
+`midline-agent/browser` monitors a web app from the page: uncaught errors and unhandled rejections,
+failed `fetch` and `XMLHttpRequest` calls, Web Vitals (LCP, INP, CLS, FCP, TTFB) and, if you ask,
+console output. It has no dependencies, no Node code, and works under React, Vue, Angular, Svelte,
+Next.js or no framework at all, because it instruments the page rather than a framework.
+
+```ts
+import * as Midline from "midline-agent/browser";
+
+Midline.init({
+  apiKey: import.meta.env.VITE_MIDLINE_BROWSER_KEY, // pk_… browser key
+  service: "checkout-web",
+  environment: import.meta.env.MODE,
+  release: import.meta.env.VITE_RELEASE,
+});
+```
+
+**Use a browser key, never a server key.** Anything in a bundle can be read by whoever loads the page,
+so browser keys (`pk_…`) are public by design and the Midline server accepts them only from the
+origins listed on the key (create one under **Project → API Keys → Browser**). Server keys (`ak_…`)
+are refused whenever a browser sends them, and the SDK won't start with one.
+
+Errors a framework catches itself never reach the window. Forward them:
+
+```tsx
+// React error boundary
+componentDidCatch(error: Error, info: ErrorInfo) {
+  Midline.captureException(error, { extra: { componentStack: info.componentStack } });
+}
+
+// Vue
+app.config.errorHandler = (error, _instance, info) => Midline.captureException(error, { extra: { info } });
+```
+
+**Linking a page to your backend.** Same-origin requests get a W3C `traceparent` header, which the
+Node agent on the backend turns into `traceId`/`spanId` (see above), so the page's failed call and
+the server request behind it share a trace id. For an API on another origin, list it in
+`tracePropagationTargets` and allow the `traceparent` header in that API's CORS configuration, or
+the browser will block the call's preflight.
+
+| Option | Default | |
+| --- | --- | --- |
+| `apiKey` | — | Browser key (`pk_…`). |
+| `endpoint` | `https://api.usemidline.com` | `http://` only for localhost. |
+| `service`, `environment`, `release` | — | Stamped on every event. |
+| `captureErrors` | `true` | Uncaught errors and unhandled rejections. |
+| `captureRequests` | `"failed"` | `"failed"` (4xx, 5xx, network errors), `"all"`, or `false`. Every call is a breadcrumb either way. |
+| `captureWebVitals` | `true` | Reported once, when the page is first hidden. |
+| `captureConsole` | `false` | `true` for `error` and `warn`, or a list of levels. Lines appear on the Terminal page. Opt-in because wrapped console calls show the SDK as their source in devtools. |
+| `tracePropagationTargets` | same origin | Strings match as URL prefixes (or path prefixes starting with `/`); RegExps match the full URL. |
+| `ignoreErrors`, `ignoreUrls` | `[]` | Strings match as substrings. |
+| `sampleRate` | `1` | Fraction of events sent. |
+| `maxEventsPerMinute` | `120` | So an error in a render loop can't flood the project. |
+| `beforeSend` | — | Return `null` to drop an event; edit `payload` and `metadata` freely. |
+| `redactFields` | — | Extra field names to redact. |
+| `enabled`, `debug` | `true`, `false` | |
+
+Also: `captureMessage(message, severity)`, `setUser({ id })`, `setTag(key, value)`,
+`addBreadcrumb(message)`, `flush()` and `close()`, which restores everything the SDK wrapped.
+
+What it will not do: it never sends cookies, query strings of the page URL, or request and response
+bodies; it redacts the same credential patterns as the Node agent before an event is queued; it
+never throws into your code; and if Midline is down it keeps a bounded queue and backs off. Pending
+events leave with `keepalive` when the tab is hidden. During server-side rendering, `init` does
+nothing.
+
+No bundler? Load the ES module build directly:
+
+```html
+<script type="module">
+  import * as Midline from "https://cdn.jsdelivr.net/npm/midline-agent@0.4.0/dist/esm/browser/index.js";
+  Midline.init({ apiKey: "pk_…", service: "web" });
+</script>
+```
+
+The Midline server must be recent enough to accept browser keys (it answers the CORS preflight on
+the ingest routes). Against an older server the SDK logs nothing unless `debug` is on and keeps
+retrying with backoff.
+
+---
+
 ## Any other backend
 
 The agent does two things: it turns a request or an error into an event, and it POSTs that event to

package/browser/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "midline-agent/browser",
+  "private": true,
+  "main": "../dist/browser/index.js",
+  "module": "../dist/esm/browser/index.js",
+  "types": "../dist/browser/index.d.ts",
+  "sideEffects": false
+}

package/dist/agent.d.ts

@@ -80,7 +80,15 @@ export declare class MidlineAgent {
     recordHttp(exchange: HttpExchange): void;
     private recordConsole;
     private takeConsoleToken;
-    /** Sends whatever is buffered now, ignoring backoff, and keeps the process alive until done. */
+    /**
+     * Sends whatever is buffered now, ignoring backoff.
+     *
+     * The transport's connection is always unref'd (telemetry alone must never
+     * keep an otherwise-idle process alive), so this relies on something else in
+     * the process keeping the event loop open until the await resolves — true for
+     * the overwhelmingly common case (a running server), not guaranteed for a
+     * bare script whose last statement is `await agent.flush()`.
+     */
     flush(): Promise<void>;
     /** Flushes with a deadline, then closes. For graceful shutdown. */
     shutdown(timeoutMs?: number): Promise<void>;

package/dist/agent.js

@@ -4,7 +4,7 @@ exports.MidlineAgent = exports.SDK_VERSION = void 0;
 const config_1 = require("./config");
 const console_1 = require("./console");
 const redact_1 = require("./redact");
-const transport_1 = require("./transport");
+const socket_transport_1 = require("./socket-transport");
 exports.SDK_VERSION = (() => {
     try {
         return require("../package.json").version;
@@ -104,15 +104,19 @@ class MidlineAgent {
             return;
         }
         this.config = resolved;
-        this.transport = new transport_1.Transport(resolved.ingestUrl, {
+        this.transport = new socket_transport_1.SocketTransport(resolved.socketOrigin, {
+            apiKey: resolved.apiKey,
             ca: resolved.ca,
             connectTimeoutMs: resolved.connectTimeoutMs,
             timeoutMs: resolved.timeoutMs,
+            reconnectionDelayMs: resolved.flushIntervalMs,
+            reconnectionDelayMaxMs: resolved.maxRetryDelayMs,
+            idleDisconnectMs: Math.max(5 * resolved.flushIntervalMs, 30000),
             userAgent: `midline-agent/${exports.SDK_VERSION} node/${process.version}`,
         });
         this.timer = setInterval(() => {
             this.consoleCapture?.flushPending();
-            void this.drain(false);
+            void this.drain();
         }, resolved.flushIntervalMs);
         // Telemetry must never be the reason a process refuses to exit.
         this.timer.unref?.();
@@ -212,7 +216,15 @@ class MidlineAgent {
         this.consoleTokens -= 1;
         return true;
     }
-    /** Sends whatever is buffered now, ignoring backoff, and keeps the process alive until done. */
+    /**
+     * Sends whatever is buffered now, ignoring backoff.
+     *
+     * The transport's connection is always unref'd (telemetry alone must never
+     * keep an otherwise-idle process alive), so this relies on something else in
+     * the process keeping the event loop open until the await resolves — true for
+     * the overwhelmingly common case (a running server), not guaranteed for a
+     * bare script whose last statement is `await agent.flush()`.
+     */
     async flush() {
         if (!this.active)
             return;
@@ -221,7 +233,7 @@ class MidlineAgent {
         if (this.drainPromise) {
             await this.drainPromise;
         }
-        await this.drain(true);
+        await this.drain();
     }
     /** Flushes with a deadline, then closes. For graceful shutdown. */
     async shutdown(timeoutMs = 5000) {
@@ -417,23 +429,23 @@ class MidlineAgent {
         }
         return batch;
     }
-    drain(keepProcessAlive) {
+    drain() {
         if (this.drainPromise)
             return this.drainPromise;
         if (!this.active || !this.queue.length || Date.now() < this.retryAfter) {
             return Promise.resolve();
         }
-        this.drainPromise = this.runDrain(keepProcessAlive).finally(() => {
+        this.drainPromise = this.runDrain().finally(() => {
             this.drainPromise = null;
         });
         return this.drainPromise;
     }
-    async runDrain(keepProcessAlive) {
+    async runDrain() {
         while (this.queue.length && this.active) {
             // Taken off the queue while in flight, so overflow trimming can't remove
             // events that are mid-send and then be confused about what was accepted.
             const batch = this.takeBatch();
-            const outcome = await this.send(batch, keepProcessAlive);
+            const outcome = await this.send(batch);
             if (outcome.kind === "ok") {
                 this.onSuccess();
                 continue;
@@ -454,7 +466,7 @@ class MidlineAgent {
                 }
                 else {
                     this.dropped += 1;
-                    this.report("http-413", "midline: the Midline server rejected an event as too large (HTTP 413); dropped it.");
+                    this.report("too-large", "midline: the Midline server rejected an event as too large; dropped it.");
                 }
                 continue;
             }
@@ -468,7 +480,7 @@ class MidlineAgent {
                 }
                 // One malformed event fails validation for the whole batch. Send them one
                 // at a time so it only costs that event.
-                const isolated = await this.sendIndividually(batch, keepProcessAlive);
+                const isolated = await this.sendIndividually(batch);
                 if (!isolated)
                     return;
                 continue;
@@ -479,16 +491,16 @@ class MidlineAgent {
         }
     }
     /** Returns false if delivery should stop for this drain. */
-    async sendIndividually(batch, keepProcessAlive) {
+    async sendIndividually(batch) {
         for (let index = 0; index < batch.length; index++) {
             if (this.consoleUnsupported && batch[index].wire.eventType === "console")
                 continue;
-            const outcome = await this.send([batch[index]], keepProcessAlive);
+            const outcome = await this.send([batch[index]]);
             if (outcome.kind === "ok") {
                 this.onSuccess();
             }
             else if (outcome.kind === "rejected" || outcome.kind === "tooLarge") {
-                const detail = outcome.kind === "rejected" ? outcome.detail : "HTTP 413";
+                const detail = outcome.kind === "rejected" ? outcome.detail : "too_large";
                 if (outcome.kind === "rejected" && this.refusedConsole(batch[index], detail))
                     continue;
                 this.dropped += 1;
@@ -506,53 +518,40 @@ class MidlineAgent {
         }
         return true;
     }
-    async send(batch, keepProcessAlive) {
-        const config = this.config;
-        const body = JSON.stringify({ events: batch.map((item) => item.wire) });
-        let result;
+    async send(batch) {
+        let ack;
         try {
-            result = await this.transport.post(config.batchUrl, body, { "x-api-key": config.apiKey }, keepProcessAlive);
+            ack = await this.transport.send(batch.map((item) => item.wire));
         }
         catch (err) {
             this.report(`transport:${errorCode(err)}`, this.describe(err, batch.length));
             return { kind: "retry" };
         }
-        const { status } = result;
-        if (status >= 200 && status < 300) {
-            // Servers that predate 401-on-bad-key answer 201 and count the rejects.
-            const summary = parseJson(result.body);
-            const failed = typeof summary?.failed === "number" ? summary.failed : 0;
-            if (failed >= batch.length && batch.length > 0) {
+        if (ack.ok) {
+            // Servers that predate 401-on-bad-key ack ok and count the rejects.
+            if (ack.rejected >= batch.length && batch.length > 0) {
                 this.log("error", "midline: the Midline server did not accept this API key. Monitoring is now off — retrying wouldn't help. Check apiKey / MIDLINE_API_KEY.");
                 return { kind: "stop" };
             }
-            if (failed > 0) {
-                this.dropped += failed;
-                this.report("partial", `midline: the Midline server rejected ${failed} of ${batch.length} events.`);
+            if (ack.rejected > 0) {
+                this.dropped += ack.rejected;
+                this.report("partial", `midline: the Midline server rejected ${ack.rejected} of ${batch.length} events.`);
             }
             return { kind: "ok" };
         }
-        if (status === 401 || status === 403) {
-            this.log("error", `midline: the Midline server rejected the API key (HTTP ${status}). ` +
+        if (ack.code === "unauthorized" || ack.code === "forbidden") {
+            this.log("error", `midline: the Midline server rejected the API key (${ack.code}). ` +
                 "Monitoring is now off — retrying wouldn't help. Check apiKey / MIDLINE_API_KEY.");
             return { kind: "stop" };
         }
-        if (status === 413) {
+        if (ack.code === "too_large") {
             return { kind: "tooLarge" };
         }
-        if (status === 408 || status === 429 || status >= 500) {
-            const retryAfterMs = parseRetryAfter(result.headers["retry-after"]);
-            this.report(`http-${status}`, `midline: the Midline server returned HTTP ${status}; events are buffered and will be retried.`);
-            return { kind: "retry", retryAfterMs };
+        if (ack.code === "rate_limited") {
+            this.report("rate_limited", `midline: the Midline server is rate-limiting this project; events are buffered and will be retried.`);
+            return { kind: "retry", retryAfterMs: ack.retryAfterMs };
         }
-        if (status >= 300 && status < 400) {
-            // Never followed: that would hand the API key to wherever the redirect points.
-            const location = String(result.headers.location ?? "").slice(0, 200);
-            this.report(`http-${status}`, `midline: the Midline endpoint redirected (HTTP ${status}${location ? ` to ${location}` : ""}). ` +
-                "Redirects are not followed; set MIDLINE_ENDPOINT to the final URL.");
-            return { kind: "retry" };
-        }
-        return { kind: "rejected", detail: `HTTP ${status}${serverMessage(result.body)}` };
+        return { kind: "rejected", detail: `${ack.code}: ${ack.message}` };
     }
     /**
      * True when a rejected event is a console line the server has no event type for,
@@ -582,7 +581,7 @@ class MidlineAgent {
     }
     describe(err, inFlight) {
         const config = this.config;
-        const origin = config.ingestUrl.origin;
+        const origin = config.socketOrigin.origin;
         const code = errorCode(err);
         const buffered = ` ${this.queue.length + inFlight} event(s) buffered; your application is unaffected.`;
         if (TLS_ERROR_REASONS[code] || code.startsWith("ERR_SSL") || code === "EPROTO") {
@@ -600,7 +599,7 @@ class MidlineAgent {
             return `midline: could not connect to ${origin} within ${config.connectTimeoutMs}ms; retrying with backoff.${buffered}`;
         }
         if (code === "ENOTFOUND" || code === "EAI_AGAIN") {
-            return `midline: cannot resolve ${config.ingestUrl.hostname} (${code}) — check MIDLINE_ENDPOINT and DNS; retrying with backoff.${buffered}`;
+            return `midline: cannot resolve ${config.socketOrigin.hostname} (${code}) — check MIDLINE_ENDPOINT and DNS; retrying with backoff.${buffered}`;
         }
         if (code === "ECONNREFUSED") {
             return `midline: ${origin} refused the connection; retrying with backoff.${buffered}`;
@@ -745,26 +744,3 @@ function errorCode(err) {
     const e = err;
     return String(e?.code ?? e?.cause?.code ?? e?.errno ?? e?.name ?? "");
 }
-function parseJson(text) {
-    try {
-        return JSON.parse(text);
-    }
-    catch {
-        return undefined;
-    }
-}
-function serverMessage(body) {
-    const message = parseJson(body)?.message;
-    const text = Array.isArray(message) ? message.join("; ") : typeof message === "string" ? message : "";
-    return text ? `: ${text.slice(0, 300)}` : "";
-}
-function parseRetryAfter(header) {
-    const value = Array.isArray(header) ? header[0] : header;
-    if (!value)
-        return undefined;
-    const seconds = Number(value);
-    if (Number.isFinite(seconds))
-        return Math.max(0, seconds * 1000);
-    const date = Date.parse(value);
-    return Number.isFinite(date) ? Math.max(0, date - Date.now()) : undefined;
-}

package/dist/browser/client.d.ts

@@ -0,0 +1,59 @@
+import type { EventSeverity } from "../types.js";
+import type { CaptureContext, MidlineBrowserConfig, MidlineUser } from "./types.js";
+export declare class BrowserConfigError extends Error {
+}
+/** Accepts a base URL or the ingest URL, and returns the batch URL. */
+export declare function resolveBatchUrl(endpoint?: string): string;
+/**
+ * One installed SDK instance. Everything it does to the page — wrapped fetch,
+ * XHR, console and history, window listeners — is undone by `close()`.
+ */
+export declare class BrowserClient {
+    private readonly win;
+    private readonly config;
+    private readonly redactor;
+    private readonly transport;
+    private readonly teardowns;
+    private readonly breadcrumbs;
+    private readonly recentErrors;
+    private readonly sessionId;
+    private readonly originalConsole;
+    private traceId;
+    private user;
+    private tags;
+    private tokens;
+    private lastRefill;
+    private active;
+    private inConsoleHook;
+    private rateLimitNoted;
+    private constructor();
+    /** Returns undefined, after one console warning, when the SDK can't run. It never throws into the app. */
+    static create(config: MidlineBrowserConfig): BrowserClient | undefined;
+    private install;
+    captureException(error: unknown, context?: CaptureContext): void;
+    captureMessage(message: string, severity?: EventSeverity, context?: CaptureContext): void;
+    setUser(user: MidlineUser | null): void;
+    setTag(key: string, value: string): void;
+    addBreadcrumb(message: string, type?: string): void;
+    flush(): Promise<void>;
+    close(): Promise<void>;
+    private handleError;
+    private handleRequest;
+    private handleConsole;
+    private handleNavigation;
+    private handleVitals;
+    private propagation;
+    private base;
+    private emit;
+    private takeToken;
+    private breadcrumb;
+    private contextPayload;
+    private applyTags;
+    /** The page path, plus a `#/…` hash route when the app routes by hash. Never the query string. */
+    private currentRoute;
+    private pageUrl;
+    private guard;
+    private debug;
+}
+/** W3C ids: lowercase hex, never all zeros. */
+export declare function randomHex(bytes: number): string;