@autter/runtime-node 1.4.0 → 1.6.0

package/README.md

@@ -5,6 +5,81 @@ application context to logs and completed operations. See the full
 [operation logging guide](../../docs/OPERATION-LOGGING.md) for initialization,
 steps, outcomes, redaction, export lifecycle, and release requirements.
 
+# Requests and coded errors (1.5.0)
+
+Requires ingester 1.5.0. One summary per request (always kept), request ids
+echoed to clients, one issue per error code:
+
+```ts
+import {
+  autterRequests, autterErrorResponse, defineRuntimeErrors, runtimeContext,
+} from "@autter/runtime-node";
+
+export const billingErrors = defineRuntimeErrors("billing", {
+  declined: { status: 402, message: "Payment declined", expected: true,
+              why: "The card issuer rejected the charge", fix: "Ask for another card" },
+  limit: ({ plan }: { plan: string }) => ({ status: 429, message: `Plan ${plan} limit reached` }),
+});
+
+app.use(autterRequests({ ignore: ["/healthz", "/metrics"] }));
+app.post("/checkout", async (req, res) => {
+  runtimeContext.set({ cart: { items: req.body.items.length } });
+  runtimeContext.info("Applied coupon", { coupon: "SPRING" });   // folded into the summary
+  if (overLimit) throw billingErrors.limit({ plan: "free" });     // Express 5 (next(err) on 4)
+  res.json({ ok: true, requestId: runtimeContext.requestId });
+});
+app.use(autterErrorResponse());   // { error: { message, code, why?, fix?, link?, requestId } }
+```
+
+Also: `autterFastify`, `withRuntimeRequest` (fetch-style handlers),
+`runtimeContext.fork` / `runInBackground` / `carrier()` for background work and
+queues, the per-operation AI usage rollup, `waitUntil`, `initAutterLogging`
+(no NodeSDK — for apps with their own OTel), enrichers (`enrichUserAgent`,
+`enrichRequestSize`, `enrichEdgeGeo`, `enrichDeployment`), sinks (`otlpSink`,
+`consoleSink`, `fileSink` → `.autter/runtime/*.jsonl` in development) and test
+helpers:
+
+```ts
+import { captureRuntime, expectOperation } from "@autter/runtime-node/testing";
+const runtime = captureRuntime();
+// … exercise the app …
+expectOperation(runtime, "POST /checkout").toHaveOutcome("degraded").toHaveErrorCode("billing.declined");
+```
+
+Full guide: [Requests, coded errors and background work](../../docs/REQUESTS-AND-ERRORS.md).
+
+# Breadcrumbs (1.6.0)
+
+The steps before an error, scoped to the current request:
+
+```ts
+import { addBreadcrumb, runtimeLogger } from "@autter/runtime-node";
+
+app.post("/checkout", async (req, res) => {
+  addBreadcrumb({ message: "Loaded cart", data: { items: 3 } });  // type "custom"
+  runtimeLogger.info("Charging card", { provider: "stripe" });     // recorded as a "log" breadcrumb
+  await charge();                                                  // a throw here carries both
+});
+```
+
+- Scope: inside `autterRequests()` / `withRuntimeRequest()` /
+  `withRuntimeOperation()` the trail belongs to the request and is shared with
+  its child operations and forks (AsyncLocalStorage). Inside a traced request
+  without the middleware it belongs to the trace; outside both, to the process
+  (so a crash outside any request carries what happened before it).
+- The last 30 are kept. `captureException`, `captureMessage` and errors thrown
+  out of operations attach them to the exception span event as
+  `autter.breadcrumbs` (a JSON string, at most 16 KB, or less when
+  `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` is lower).
+- Every `runtimeLogger` / `runtimeContext.debug|info|warn|error` entry is
+  recorded as a `log` breadcrumb with its level and up to 10 primitive
+  attributes.
+- Messages (≤ 500 characters) and data (≤ 10 keys of strings, numbers,
+  booleans or null) are scrubbed with your `redact` settings.
+- The browser relay forwards browser breadcrumbs, re-validated and re-scrubbed.
+- Needs ingester 1.6.0. Logger-only mode (`initAutterLogging`) and
+  `@autter/runtime-edge` don't attach them yet.
+
 # @autter/runtime-node
 
 The server tracker exports per-instance RSS, heap, memory limit, and GC
@@ -70,6 +145,12 @@ const autter = initAutterServer({
 // handled errors — always recorded, never sampled out:
 autter.captureException(err, { "order.id": "…" });
 
+// fingerprint override — every error carrying the same fingerprint is one
+// issue, whatever its message or stack (a string or up to 10 strings, each
+// ≤200 chars; sent as `autter.fingerprint` on the exception event; needs
+// ingester 1.6.0+, older ingesters ignore it):
+autter.captureException(err, { fingerprint: ["checkout", "card-timeout"] });
+
 // warnings/info without an exception — same grouping+aggregation as
 // errors, just a lower severity ("fatal" | "error" | "warning" | "info"):
 autter.captureMessage("Legacy /orders lookup used", "warning");
@@ -99,7 +180,7 @@ Defaults (cheap by construction):
 | Traces containing an error | 100% (tail retention, `retainTracesOnError`) |
 | `withProcessSpan` spans | 100% (same always-on tracer) |
 | LLM/GenAI call spans | 100% (`llmTracing`, on by default) |
-| Healthy traces | 1% head sampling (`traceSampleRate`) |
+| Healthy traces | 1% head sampling (`traceSampleRate`; also for requests whose incoming `traceparent` is unsampled, such as browser fetches from `@autter/runtime-browser`) |
 | Request metrics | exported every 60 s |
 | Logs | not collected |
 | PII in custom attributes | redacted before export (`redactAttributes`) |
@@ -124,6 +205,92 @@ import { ExpressInstrumentation } from "@opentelemetry/instrumentation-express";
 initAutterServer({ ..., instrumentations: [new ExpressInstrumentation()] });
 ```
 
+### Server errors keep their stacks
+
+When a handler throws, the framework answers 500 — and on its own that 500
+is all the tracker sees: a stackless `HttpServerError` grouped by route. Add
+your framework's hook and the thrown error is recorded, with its stack, on
+the request span; the framework then carries on exactly as before.
+
+```ts
+import { autterErrorHandler, autterFastifyErrors, autterKoa, captureRequestError } from "@autter/runtime-node";
+
+// Express / Connect: after your routes, before your own error handlers.
+// Records the error, then calls next(err) unchanged.
+app.use(autterErrorHandler());
+
+// Fastify 4/5: an onError hook (Fastify's error handler still answers).
+// autterFastify (request summaries) already includes it.
+app.register(autterFastifyErrors);   // or autterFastifyErrors(app)
+
+// Koa 2/3: mount first; records and rethrows, so app.on("error") still fires.
+app.use(autterKoa());
+
+// Anything else (NestJS exception filters, hapi onPreResponse, …):
+captureRequestError(err, { "app.handler": "checkout" });
+```
+
+All three report errors with status ≥ 500 (or none), RuntimeErrors and coded
+errors — `ctx.throw(404)` and validation failures stay out of your issues.
+Pass `{ capture: true }` to report every error, or a predicate. An error is
+reported once, even if your handler already called `captureException`, and it
+lands on the request's HTTP server span even when an instrumentation's child
+span is active (`captureRequestError` still adds its attributes to that span
+when the error was already reported). Errors from another JavaScript realm
+(a `vm` context, Next's edge sandbox under `next start`) keep their own type,
+message and stack. `autterErrorResponse()` reports the same way and also
+answers the request.
+
+### Unhandled promise rejections
+
+Recorded as errors (severity `error`, `autter.unhandled_rejection: true`, not
+fatal) — without changing whether your process crashes. Any
+`unhandledRejection` listener switches off Node's default crash, so the SDK
+does what Node would have done when its listener is the only one:
+
+| `--unhandled-rejections` | Your own listener? | What happens | Recorded as |
+| --- | --- | --- | --- |
+| `throw` (default) | no | rethrown on the next tick: exit code 1, as before | fatal crash (`autter.unhandled`) |
+| `throw` (default) | yes | your listener decides | error |
+| `strict` | either | Node crashes first, as before | fatal crash |
+| `warn`, `none` | either | no crash, as before | error |
+| `warn-with-error-code` | no | Node's warning and exit code 1, as before | error |
+
+The one visible difference: after the rethrow, `uncaughtException` listeners
+receive origin `"uncaughtException"` rather than `"unhandledRejection"`.
+`captureUnhandledRejections: false` (or `captureGlobalErrors: false`) restores
+the 1.5.0 behaviour, where a rejection is seen only when it crashes the
+process.
+
+### Version compatibility: checked for you
+
+Some features need a minimum ingester version: operation logging needs
+1.4.0, memory metrics 1.3.3, endpoint latency 1.3.1. After
+`initAutterServer`, the SDK checks the ingester once in the background
+(`GET /v1/compat`, unref'd, 3 s timeout). For each feature in use that the
+ingester can't store, it prints one warning:
+
+```text
+[autter-runtime] Operation logging needs ingester >= 1.4.0; yours is 1.3.4. Upgrade the ingester: docker pull ghcr.io/autter-dev/otlp-ingester:latest and restart it (ClickHouse migrations run at boot). See …/docs/COMPATIBILITY.md
+```
+
+The check never throws, never delays startup or exit, and is silent when
+versions match or are unknown. `debug: true` logs the result. Turn it off
+with `compatCheck: false` or `AUTTER_COMPAT_CHECK=0`. The browser relay does
+the same for browser features such as CSP violation capture. The SDK reports
+itself through the OTLP resource attributes `telemetry.distro.name` and
+`telemetry.distro.version`, so the Autter dashboard shows the SDK version
+each service runs.
+
+For a full report (for example in CI or after a deploy):
+
+```bash
+npx @autter/runtime-node doctor --endpoint https://ingest.example.com [--key "$AUTTER_RUNTIME_KEY"] [--json]
+```
+
+Exit codes: 0 compatible, 1 mismatch or rejected key, 2 ingester unreachable.
+See [docs/COMPATIBILITY.md](../../docs/COMPATIBILITY.md).
+
 ### Lifecycle: never lose telemetry to a forgotten shutdown()
 
 Telemetry is batched (errors every ~2 s, healthy traces every ~5 s, metrics
@@ -155,23 +322,39 @@ default 1% sampling the span contribution is negligible; if you set
 Tail-retained error traces don't distort this: their spans carry
 `autter.tail_retained` and the ingester keeps them out of span-fed rollups.
 
-### Privacy: custom attributes are redacted before they leave the process
-
-The browser relay whitelist-sanitises everything a client posts — but the
-server tracker accepts free-form attributes from *your* code, where a stray
-`captureException(err, { "user.email": … })` would otherwise go out
-verbatim. Custom attributes are therefore scrubbed at capture time by
-default:
-
-- values that look like emails, JWTs, `sk-…`/`ghp_…`/AWS/Slack tokens,
-  `Bearer …` headers, or `scheme://user:pass@host` URLs are masked;
+### Privacy: secrets and PII are scrubbed before they leave the process
+
+Error context is where secrets leak: `connect ECONNREFUSED
+postgres://admin:hunter2@db…` in an exception message, a provider error
+echoing an API key, a `?token=` in a request URL, a stray
+`captureException(err, { "user.email": … })`. Everything the SDK exports is
+therefore scrubbed by default — custom attributes, exception messages and
+stack traces, span status messages, LLM call attributes and errors,
+structured logs, and the browser events the relay forwards. A final pass at
+export time scrubs every span regardless of who created it (HTTP
+instrumentation URLs, third-party instrumentations, `recordException` in
+your own code).
+
+- secrets inside any string are masked in place, keeping the rest of the
+  message/stack readable: JWTs, `Bearer`/`Basic` credentials,
+  `Authorization:`/`Cookie:`/`Set-Cookie:` header text,
+  `scheme://user:pass@host` connection strings (postgres, mysql,
+  mongodb+srv, redis, …), vendor keys (`sk-…`, `sk_live_…`, `ghp_…`,
+  `github_pat_…`, `xox?-…`, `AIza…`, `AKIA…`, …), PEM private keys,
+  `password=`/`?token=`/`?api_key=`-style assignments, emails, and
+  Luhn-valid card numbers;
 - attributes whose **key** looks sensitive (`password`, `token`, `secret`,
-  `api_key`, `authorization`, `cookie`, `ssn`, `card_number`, …) are masked
-  wholesale;
-- non-sensitive keys and primitives pass through untouched, so grouping and
-  dashboards keep working.
+  `api_key`, `authorization`, `cookie`, `session`, `ssn`, `card_number`, …)
+  are masked wholesale, at any nesting depth;
+- non-sensitive keys, numbers (including LLM token counts) and booleans pass
+  through untouched, so grouping, dashboards and cost tracking keep working.
+
+The same patterns run in the browser SDK, the relay, the Python adapter,
+and again in the ingester before storage (shared test vectors in
+`test-vectors/redaction.json`).
 
-Disable or extend it per service:
+Extend it (applies to messages, stacks and URLs too) or disable it per
+service:
 
 ```ts
 initAutterServer({
@@ -193,7 +376,9 @@ const safe = makeSafeCapture();
 safe.captureException(err, { "user.email": email }); // masked before export
 ```
 
-The raw primitive is exported too (`redactAttributes(attrs, options)`).
+The raw primitives are exported too (`redactAttributes(attrs, options)`,
+`redactText(text, options)`). The browser relay takes the same options as
+`createBrowserRelayHandler({ ..., redact })`.
 This closes the server-side gap to match the browser relay's payload
 whitelist; it is best-effort scrubbing of obvious PII shapes, not a DLP
 engine — keep secrets out of attributes in the first place.