@wtfalch/reporting 0.5.1 → 0.8.1

package/README.md

@@ -25,24 +25,99 @@ Published: yes, v0.5.1, 2026-09-28
 ## Install
 
 ```sh
-pnpm add @wtfalch/reporting
-pnpm exec reporting-migrations   # copies migrations/*.sql into drizzle/ as the next numbers
+pnpm add @wtfalch/reporting @wtfalch/db
 ```
 
-The copied migration creates `reporting_events`, `reporting_tasks` and
-`reporting_settings`, a `SECURITY DEFINER` prune function, and, when a role
-named `<database>_rt` exists, revokes UPDATE, DELETE and TRUNCATE on events
-from it. That is the estate's runtime-role convention; a host without the
-role gets the tables and the function and no revokes take effect. Your
-migrate script applies the file; this package never connects to apply
-anything.
+The package ships SQL and never connects or migrates. The host installs
+`@wtfalch/db`, applies the SQL with the owner credential into a schema of
+its own, then makes the runtime role. The runtime `searchPath` and
+`ensureRuntimeRole({ schemas })` must both cover that schema. The SQL names
+no schema and no role: its `SECURITY DEFINER` functions use `set search_path
+from current`, so they find their tables in whatever schema the host
+migrated into. Migrate with `@wtfalch/db` 0.5.2 or later: 0.5.0 and 0.5.1
+let a temp table shadow a table inside a `SECURITY DEFINER` function, and
+0.5.2 puts `pg_temp` last on the runner's `search_path`.
+
+```ts
+import { migrationsDir as reportingMigrations } from '@wtfalch/reporting/migrations-dir';
+import { runMigrationSources } from '@wtfalch/db/migrate';
+import { ensureRuntimeRole } from '@wtfalch/db/runtime-role';
+
+await runMigrationSources({
+  url: ownerUrl,
+  schema: 'myservice', // created if missing; the tables land here, never in public
+  sources: [
+    { name: 'reporting', dir: reportingMigrations }, // before the host's own
+    { name: 'app', dir: 'drizzle' },
+  ],
+});
+
+await ensureRuntimeRole({
+  ownerUrl,
+  runtimeUrl,
+  schemas: ['myservice'],
+  appendOnly: ['reporting_events', 'reporting_analytics'], // insert and read only
+  readOnly: ['reporting_analytics_daily', 'reporting_analytics_weekly'], // the rollups: written only by the functions
+  noDelete: [ // insert and update, never delete (needs @wtfalch/db 0.5.2 or later)
+    'reporting_tasks',
+    'reporting_settings',
+    'reporting_errors',
+    'reporting_tenant_settings',
+    'reporting_error_tenants',
+    'reporting_requests',
+  ],
+  grants: [ // the seven security definer functions, schema-qualified
+    'myservice.reporting_prune_events(interval, integer)',
+    'myservice.reporting_rollup_day(date)',
+    'myservice.reporting_rollup_week(date)',
+    'myservice.reporting_prune_analytics(interval, integer, date)',
+    'myservice.reporting_erase_person(text)',
+    'myservice.reporting_prune_errors(interval, integer)',
+    'myservice.reporting_prune_open_errors(integer, integer)',
+  ],
+});
+```
+
+Run `ensureRuntimeRole` after every migration run: the lists apply to the
+tables that exist at that moment. `grants` entries are spliced into SQL, so
+they are trusted text. Without `noDelete` the role can delete from those six
+tables. These are the privileges the
+package's own tests check, on a real Postgres, in a named schema.
+
+At boot, check the role over the runtime connection, whose `searchPath`
+includes the schema:
+
+```ts
+import { assertRuntimeRole } from '@wtfalch/db/runtime-role';
+
+await assertRuntimeRole(connection.database, {
+  appendOnly: ['myservice.reporting_events', 'myservice.reporting_analytics'],
+  readOnly: ['myservice.reporting_analytics_daily', 'myservice.reporting_analytics_weekly'],
+  noDelete: [
+    'myservice.reporting_tasks',
+    'myservice.reporting_settings',
+    'myservice.reporting_errors',
+    'myservice.reporting_tenant_settings',
+    'myservice.reporting_error_tenants',
+    'myservice.reporting_requests',
+  ],
+});
+```
+
+The runtime handle is the native Drizzle one: pass `withDrizzle(runtime, {
+schema: tables }).orm` (or `tx.orm`) from `@wtfalch/db/drizzle`.
+
+Without `@wtfalch/db`, `reporting-migrations` still copies
+`migrations/*.sql` into `drizzle/` as the next numbers and the host applies
+them itself. Prefer `runMigrationSources`: the copy path has no schema and no
+runtime-role step, so the host must grant and revoke by hand.
 
 ## Bind it once
 
 ```ts
 // src/lib/reporting/core.ts, the one file that imports your raw handle
 import { createReporting } from '@wtfalch/reporting';
-import { createHousekeeping, pruneEvents, retentionLag } from '@wtfalch/reporting/housekeeping';
+import { createHousekeeping, pruneErrors, pruneEvents, retentionLag } from '@wtfalch/reporting/housekeeping';
 import { deferWithAfter } from '@wtfalch/reporting/next';
 
 export const reporting = createReporting({
@@ -55,6 +130,7 @@ export const reporting = createReporting({
 
 export const housekeeping = createHousekeeping({ reporting });
 housekeeping.register(pruneEvents);
+housekeeping.register(pruneErrors);
 housekeeping.register(retentionLag);
 ```
 
@@ -131,6 +207,31 @@ edit can shorten history to a week and no further. Pass `audit` to
 `createReporting` to have every change handed to your audit ledger after it
 commits.
 
+## Release health (0.8.0)
+
+With `REPORTING_RELEASE` (an image digest) and `REPORTING_COLOR` (`blue` or
+`green`) set, every event, analytics and error row carries both. Count each
+response and mount the read foundry judges a new color by:
+
+```ts
+// app/api/reporting/release-health/route.ts
+import { releaseHealthHandler } from '@wtfalch/reporting/next';
+export const GET = releaseHealthHandler({ db, reporting });
+
+// any route handler: counted per minute, color and release (status >= 500 is an error)
+import { withRequestCount } from '@wtfalch/reporting/next';
+export const POST = withRequestCount(reporting, async (req) => Response.json({ ok: true }));
+```
+
+`reporting.requests.record(status)` is the same count for a host that has no
+route handler to wrap. Counts sit in memory and are written once per minute
+bucket (every 10 seconds by default; `reporting.flush()` writes them now). The
+read needs `Authorization: Bearer $REPORTING_HEALTH_TOKEN`: 401 for a missing or
+wrong token, 503 when the variable is unset. It answers `from` and `to` (ISO,
+`from` before `to`, at most 8 days; otherwise 400) with request and 5xx counts
+per color and the per-day average of the 7 days before `from`. Counts only; no
+event content. Rows of `reporting_requests` are not pruned.
+
 ## Shutdown
 
 ```ts
@@ -217,11 +318,10 @@ the rollups hold nothing to erase. Call it in the host's erasure transaction.
 
 ## Tests
 
-```ts
-import { createFakeReporting } from '@wtfalch/reporting/fake';
-```
-
-Same validation, no database; `fake.rows` is the table.
+No package ships a stand-in for a service. `createFakeReporting` (same
+validation, no database, `fake.rows` is the table) is this package's own
+unexported test double, under `src/test/`, for its own test files only -- a
+host writes its own double, or runs this package against a real database.
 
 ## What this package never does
 

package/dist/analytics/write.js

@@ -1,7 +1,9 @@
+import { deployTags } from '../deploy.js';
 import { reportingAnalytics } from '../tables.js';
 export async function insertAnalytics(db, rows) {
     if (rows.length === 0)
         return;
+    const tags = deployTags();
     await db.insert(reportingAnalytics).values(rows.map((r) => ({
         occurredAt: r.occurredAt,
         receivedAt: r.receivedAt,
@@ -16,5 +18,6 @@ export async function insertAnalytics(db, rows) {
         device: r.device,
         country: r.country,
         props: r.props,
+        ...tags,
     })));
 }

package/dist/deploy.d.ts

@@ -0,0 +1,11 @@
+export type Color = 'blue' | 'green';
+export interface DeployTags {
+    readonly release: string | null;
+    readonly color: Color | null;
+}
+/**
+ * The release and color the platform gave this container
+ * (`REPORTING_RELEASE`, `REPORTING_COLOR`). A color that is not blue or green
+ * is null, so a bad value cannot fail a write on the column's CHECK.
+ */
+export declare function deployTags(env?: NodeJS.ProcessEnv): DeployTags;

package/dist/deploy.js

@@ -0,0 +1,13 @@
+/**
+ * The release and color the platform gave this container
+ * (`REPORTING_RELEASE`, `REPORTING_COLOR`). A color that is not blue or green
+ * is null, so a bad value cannot fail a write on the column's CHECK.
+ */
+export function deployTags(env = process.env) {
+    const color = env.REPORTING_COLOR;
+    const release = env.REPORTING_RELEASE?.slice(0, 128);
+    return {
+        release: release ? release : null,
+        color: color === 'blue' || color === 'green' ? color : null,
+    };
+}

package/dist/errors/capture.js

@@ -1,9 +1,10 @@
 import { eq, sql } from 'drizzle-orm';
+import { deployTags } from '../deploy.js';
 import { ERROR_RUNTIMES, KIND_PATTERN, LIMITS } from '../schema.js';
-import { reportingErrors } from '../tables.js';
+import { reportingErrorTenants, reportingErrors } from '../tables.js';
 import { describe } from '../writer.js';
 import { errorKind, fingerprint } from './fingerprint.js';
-import { heldSecrets, redactText } from './redact.js';
+import { heldSecrets, redactPii, redactText } from './redact.js';
 const CAMEL_BOUNDARY = /([a-z0-9])([A-Z])/g;
 const INVALID_KIND_CHARS = /[^a-z0-9_]+/g;
 /** camelCase/PascalCase/anything, folded down to what `KIND_PATTERN` allows. */
@@ -51,6 +52,8 @@ function messageOf(error, kind) {
 function stackOf(error) {
     return error instanceof Error && typeof error.stack === 'string' ? error.stack : null;
 }
+/** Extra characters kept before redaction so a secret at the size limit is seen whole. */
+const SLACK = 1024;
 /** Keeps the top of the text — the start, where the throw site and its callers are — and drops the rest. */
 function truncateEnd(text, max) {
     return text.length > max ? text.slice(0, max) : text;
@@ -101,7 +104,7 @@ function normaliseRuntime(runtime) {
  * share one transaction so the lock from the first is still held for the
  * second.
  */
-async function upsertGroup(db, site, s) {
+async function upsertGroup(db, site, s, log) {
     return db.transaction(async (tx) => {
         const prior = await tx
             .select({ state: reportingErrors.state })
@@ -120,6 +123,7 @@ async function upsertGroup(db, site, s) {
             stack: s.stack,
             runtime: s.runtime,
             release: s.release,
+            color: s.color,
             environment: s.environment,
             firstSeenAt: s.at,
             lastSeenAt: s.at,
@@ -131,12 +135,13 @@ async function upsertGroup(db, site, s) {
             target: reportingErrors.fingerprint,
             set: {
                 occurrences: sql `${reportingErrors.occurrences} + 1`,
-                lastSeenAt: s.at,
+                lastSeenAt: sql `greatest(${reportingErrors.lastSeenAt}, ${s.at.toISOString()}::timestamptz)`,
                 message: s.message,
                 stack: s.stack,
                 tenantId: s.tenantId,
                 requestId: s.requestId,
                 release: s.release,
+                color: s.color,
                 environment: s.environment,
                 state: reopened,
                 resolvedAt: sql `case when ${reportingErrors.state} in ('resolved', 'ignored') then null else ${reportingErrors.resolvedAt} end`,
@@ -147,6 +152,33 @@ async function upsertGroup(db, site, s) {
             // false when it took the on-conflict-update path -- the standard,
             // race-free way to tell the two apart from an upsert's own result.
             .returning({ created: sql `(xmax = 0)` });
+        if (s.tenantId) {
+            // In a savepoint and fail-soft: a host that has not applied 0008 yet
+            // must still get the group row. The tenant row is the lesser loss.
+            try {
+                await tx.transaction(async (sp) => {
+                    await sp
+                        .insert(reportingErrorTenants)
+                        .values({
+                        fingerprint: s.fingerprint,
+                        tenantId: s.tenantId,
+                        occurrences: 1,
+                        firstSeenAt: s.at,
+                        lastSeenAt: s.at,
+                    })
+                        .onConflictDoUpdate({
+                        target: [reportingErrorTenants.fingerprint, reportingErrorTenants.tenantId],
+                        set: {
+                            occurrences: sql `${reportingErrorTenants.occurrences} + 1`,
+                            lastSeenAt: sql `greatest(${reportingErrorTenants.lastSeenAt}, ${s.at.toISOString()}::timestamptz)`,
+                        },
+                    });
+                });
+            }
+            catch (err) {
+                log.warn({ err: describe(err), fingerprint: s.fingerprint }, 'reporting: tenant row not written; is migration 0008 applied?');
+            }
+        }
         if (written[0]?.created)
             return 'created';
         return priorState === 'resolved' || priorState === 'ignored' ? 'reopened' : 'unchanged';
@@ -171,9 +203,15 @@ export function createCapture(o) {
         try {
             const kind = resolveKind(error, context.kind);
             const secrets = heldSecrets(process.env, o.redactEnvVars);
-            const message = truncateEnd(redactText(messageOf(error, kind), secrets), LIMITS.message);
+            // Bound the text before redacting it: the patterns must never see an
+            // uncapped, attacker-sized string. The cut keeps slack past the limit so
+            // a secret straddling the limit is redacted whole, then the final cut
+            // applies the real limit.
+            const message = truncateEnd(redactPii(redactText(truncateEnd(messageOf(error, kind), LIMITS.message + SLACK), secrets)), LIMITS.message);
             const rawStack = stackOf(error);
-            const stack = rawStack === null ? null : truncateEnd(redactText(rawStack, secrets), LIMITS.stack);
+            const stack = rawStack === null
+                ? null
+                : truncateEnd(redactPii(redactText(truncateEnd(rawStack, LIMITS.stack + SLACK), secrets)), LIMITS.stack);
             const fp = fingerprint({ kind, message, stack });
             const runtime = normaliseRuntime(context.runtime);
             // A tenant id that is not a uuid would be refused by the event row's
@@ -202,13 +240,14 @@ export function createCapture(o) {
                 stack,
                 runtime,
                 release,
+                color: deployTags().color,
                 environment,
                 tenantId,
                 requestId,
                 at,
             };
             try {
-                o.defer(() => upsertGroup(o.db, o.site, sample)
+                o.defer(() => upsertGroup(o.db, o.site, sample, o.log)
                     .then((outcome) => {
                     if (outcome === 'unchanged' || !o.alert)
                         return;

package/dist/errors/reader.d.ts

@@ -1,5 +1,5 @@
 import type { ErrorState } from '../schema.js';
-import type { ReportingErrorRow } from '../tables.js';
+import type { ReportingErrorRow, TenantErrorRow } from '../tables.js';
 import type { Actor, Db, ErrorsPage, ErrorsPageOptions } from '../types.js';
 /**
  * Newest first, keyset-paged on `(last_seen_at, fingerprint)`, the table's
@@ -11,9 +11,28 @@ import type { Actor, Db, ErrorsPage, ErrorsPageOptions } from '../types.js';
  * for its scale. No permission is applied here: the host gates, and this
  * returns rows as stored.
  */
+export declare function errorsPage(db: Db, opts?: ErrorsPageOptions & {
+    readonly tenantId?: undefined;
+}): Promise<ErrorsPage<ReportingErrorRow>>;
+export declare function errorsPage(db: Db, opts: ErrorsPageOptions & {
+    readonly tenantId: string;
+}): Promise<ErrorsPage<TenantErrorRow>>;
 export declare function errorsPage(db: Db, opts?: ErrorsPageOptions): Promise<ErrorsPage>;
-/** One row by fingerprint, or null when it is unknown. */
-export declare function errorDetail(db: Db, fingerprint: string): Promise<ReportingErrorRow | null>;
+/**
+ * One row by fingerprint, or null when it is unknown. With `tenantId`, null
+ * too when that tenant never hit it, and the row is a `TenantErrorRow`: the
+ * counts and times are the tenant's own and the shared text and triage state
+ * are null, as in `errorsPage`. A present `tenantId` that is not a uuid throws.
+ */
+export declare function errorDetail(db: Db, fingerprint: string, opts?: {
+    readonly tenantId?: undefined;
+}): Promise<ReportingErrorRow | null>;
+export declare function errorDetail(db: Db, fingerprint: string, opts: {
+    readonly tenantId: string;
+}): Promise<TenantErrorRow | null>;
+export declare function errorDetail(db: Db, fingerprint: string, opts?: {
+    readonly tenantId?: string;
+}): Promise<ReportingErrorRow | TenantErrorRow | null>;
 /**
  * Moves an error's state. 'resolved' and 'ignored' stamp `resolved_at` and
  * `resolved_by` with this instant and who did it; moving back to 'open'

package/dist/errors/reader.js

@@ -1,32 +1,76 @@
 import { and, desc, eq, ilike, lt, or, sql } from 'drizzle-orm';
-import { reportingErrors } from '../tables.js';
+import { reportingErrorTenants, reportingErrors } from '../tables.js';
 /** Postgres's default LIKE/ILIKE escape is backslash; escaping the pattern's own three special characters is what makes a search term match itself literally rather than as a wildcard pattern. */
 function likePattern(term) {
     return `%${term.replace(/[\\%_]/g, (c) => `\\${c}`)}%`;
 }
+const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
 /**
- * Newest first, keyset-paged on `(last_seen_at, fingerprint)`, the table's
- * index. Every filter is an equality on an indexed column, except `search`:
- * a case-insensitive substring match on `message` or `stack`, for triaging
- * an incident by grepping the group list rather than raw rows by hand (gap
- * issue #8). Unindexed by design -- a trigram index would need the
- * pg_trgm extension, an operational cost this triage tool does not carry
- * for its scale. No permission is applied here: the host gates, and this
- * returns rows as stored.
+ * Scope is decided by whether the key is present, not by truthiness: an empty
+ * string or an explicit `undefined` from a host bug must not fall through to
+ * the unscoped read of every tenant's data, so a present key that is not a
+ * uuid throws. An absent key is unscoped.
  */
+function tenantScope(opts) {
+    if (!('tenantId' in opts))
+        return undefined;
+    const { tenantId } = opts;
+    if (typeof tenantId !== 'string' || !UUID.test(tenantId)) {
+        throw new Error('tenantId must be a uuid when given; refusing to fall back to an unscoped read');
+    }
+    return tenantId;
+}
+/**
+ * The columns a tenant may see of a shared group. The text and request facts
+ * of the group belong to whichever tenant hit it last, and its triage state is
+ * one operator's call for every tenant, so those are null.
+ */
+function tenantColumns() {
+    return {
+        fingerprint: reportingErrors.fingerprint,
+        site: reportingErrors.site,
+        environment: reportingErrors.environment,
+        kind: reportingErrors.kind,
+        message: sql `null`.as('message'),
+        stack: sql `null`.as('stack'),
+        runtime: sql `null`.as('runtime'),
+        release: sql `null`.as('release'),
+        color: sql `null`.as('color'),
+        requestId: sql `null`.as('request_id'),
+        resolvedBy: sql `null`.as('resolved_by'),
+        state: sql `null`.as('state'),
+        resolvedAt: sql `null`.as('resolved_at'),
+        tenantId: reportingErrorTenants.tenantId,
+        occurrences: reportingErrorTenants.occurrences,
+        firstSeenAt: reportingErrorTenants.firstSeenAt,
+        lastSeenAt: reportingErrorTenants.lastSeenAt,
+    };
+}
 export async function errorsPage(db, opts = {}) {
+    const tenantId = tenantScope(opts);
     const limit = Math.max(1, Math.min(opts.limit ?? 50, 200));
+    // With a tenant, the page is that tenant's own view: its rows join the
+    // group and supply the counts and times, so ordering and paging follow the
+    // tenant's last occurrence, not anyone else's.
+    const seen = tenantId !== undefined ? reportingErrorTenants.lastSeenAt : reportingErrors.lastSeenAt;
     const conditions = [];
     if (opts.site)
         conditions.push(eq(reportingErrors.site, opts.site));
-    if (opts.state)
+    // `state` is shared operator triage, so in tenant scope it is ignored: a
+    // filter on it would show one tenant whether another reopened the group.
+    if (opts.state && tenantId === undefined)
         conditions.push(eq(reportingErrors.state, opts.state));
-    if (opts.runtime)
+    // `runtime` and the text are the latest sample from any tenant, so a tenant
+    // scope neither filters on the first nor searches the rest: either would be
+    // an oracle on another tenant's data. Search matches `kind` there.
+    if (opts.runtime && tenantId === undefined)
         conditions.push(eq(reportingErrors.runtime, opts.runtime));
     const search = opts.search?.trim().slice(0, 200);
     if (search) {
         const pattern = likePattern(search);
-        const clause = or(ilike(reportingErrors.message, pattern), ilike(reportingErrors.stack, pattern));
+        const clause = tenantId !== undefined
+            ? ilike(reportingErrors.kind, pattern)
+            : or(ilike(reportingErrors.message, pattern), ilike(reportingErrors.stack, pattern));
         if (clause)
             conditions.push(clause);
     }
@@ -34,16 +78,26 @@ export async function errorsPage(db, opts = {}) {
         conditions.push(eq(reportingErrors.environment, opts.environment));
     if (opts.after) {
         const { lastSeenAt, fingerprint } = opts.after;
-        const keyset = or(lt(reportingErrors.lastSeenAt, lastSeenAt), and(eq(reportingErrors.lastSeenAt, lastSeenAt), lt(reportingErrors.fingerprint, fingerprint)));
+        const keyset = or(lt(seen, lastSeenAt), and(eq(seen, lastSeenAt), lt(reportingErrors.fingerprint, fingerprint)));
         if (keyset)
             conditions.push(keyset);
     }
-    const rows = await db
-        .select()
-        .from(reportingErrors)
-        .where(conditions.length > 0 ? and(...conditions) : undefined)
-        .orderBy(desc(reportingErrors.lastSeenAt), desc(reportingErrors.fingerprint))
-        .limit(limit + 1);
+    const where = conditions.length > 0 ? and(...conditions) : undefined;
+    const order = [desc(seen), desc(reportingErrors.fingerprint)];
+    const rows = tenantId !== undefined
+        ? await db
+            .select(tenantColumns())
+            .from(reportingErrors)
+            .innerJoin(reportingErrorTenants, and(eq(reportingErrorTenants.fingerprint, reportingErrors.fingerprint), eq(reportingErrorTenants.tenantId, tenantId)))
+            .where(where)
+            .orderBy(...order)
+            .limit(limit + 1)
+        : await db
+            .select()
+            .from(reportingErrors)
+            .where(where)
+            .orderBy(...order)
+            .limit(limit + 1);
     const items = rows.slice(0, limit);
     const last = items.at(-1);
     return {
@@ -53,8 +107,17 @@ export async function errorsPage(db, opts = {}) {
             : null,
     };
 }
-/** One row by fingerprint, or null when it is unknown. */
-export async function errorDetail(db, fingerprint) {
+export async function errorDetail(db, fingerprint, opts = {}) {
+    const tenantId = tenantScope(opts);
+    if (tenantId !== undefined) {
+        const scoped = await db
+            .select(tenantColumns())
+            .from(reportingErrors)
+            .innerJoin(reportingErrorTenants, and(eq(reportingErrorTenants.fingerprint, reportingErrors.fingerprint), eq(reportingErrorTenants.tenantId, tenantId)))
+            .where(eq(reportingErrors.fingerprint, fingerprint))
+            .limit(1);
+        return scoped[0] ?? null;
+    }
     const rows = await db
         .select()
         .from(reportingErrors)

package/dist/errors/redact.d.ts

@@ -42,6 +42,7 @@ export declare function heldSecrets(env: Readonly<Record<string, string | undefi
  * than wrapping each in an object just to hand it to `scrub`.
  */
 export declare function redactText(text: string, secrets: readonly string[]): string;
+export declare function redactPii(text: string): string;
 /**
  * The same structure with every held secret replaced.
  *

package/dist/errors/redact.js

@@ -68,6 +68,69 @@ export function redactText(text, secrets) {
     }
     return out;
 }
+/**
+ * Pattern-based PII redaction for a captured `message` or `stack`.
+ *
+ * `redactText` only knows the exact strings the host holds. A user's email
+ * address or a bearer token inside an error message is not one of them, yet
+ * it would be stored verbatim, kept until pruned and readable by anyone who
+ * can open the errors page. This covers the common shapes by pattern: cheap,
+ * and with false negatives (a bare opaque token with no label is not caught).
+ * A string that only looks like a secret and is not one, such as a version
+ * like `lodash@4.17.20`, is left alone.
+ */
+const PII_PATTERNS = [
+    // URL userinfo: `scheme://user:pass@host` keeps scheme and user, drops the
+    // password (also `scheme://:pass@host`). Before the email rule, which would
+    // otherwise take `pass@host` and leave the user behind a wrong label.
+    [/\b([a-z][a-z0-9+.-]{0,30}:\/\/[^\s:@/]{0,64}:)[^\s@/]{1,256}@/gi, '$1[redacted: password]@'],
+    // Every quantifier is bounded (RFC 5321 limits: 64-char local part, 63-char
+    // labels), so a long run of `a.a.a.` with no `@` costs a fixed amount per
+    // start position rather than a rescan of the rest of the string.
+    [
+        /\b[A-Za-z0-9._%+-]{1,64}@[A-Za-z0-9-]{1,63}(?:\.[A-Za-z0-9-]{1,63}){0,8}\.[A-Za-z]{2,24}\b/g,
+        '[redacted: email]',
+    ],
+    // A PEM private key block; an unterminated one (a truncated stack) runs to the end.
+    [
+        /-----BEGIN [A-Z ]{0,30}PRIVATE KEY-----[\s\S]*?(?:-----END [A-Z ]{0,30}PRIVATE KEY-----|$)/g,
+        '[redacted: private key]',
+    ],
+    // The JSON form of an Authorization header, whatever the value's length.
+    [/("authorization"\s*:\s*)"(?:[^"\\]|\\.)*"/gi, '$1"[redacted: token]"'],
+    // A Cookie or Set-Cookie header's value, to the end of the line.
+    [/\b((?:set-)?cookie\s*:\s*)[^\r\n]+/gi, '$1[redacted: cookie]'],
+    // A JSON "cookie" / "set-cookie" value.
+    [/("(?:set-)?cookie"\s*:\s*)"(?:[^"\\]|\\.){0,2048}"/gi, '$1"[redacted: cookie]"'],
+    // A labelled secret in `=`, `:`, JSON (`"password":"x"`) or `%3D` form,
+    // including `x-api-key:` headers. A quoted value is taken whole.
+    [
+        /((?:password|passwd|token|secret|api[_-]?key|apikey|client_secret|access_token|refresh_token)["']?\s*(?::|=|%3D)\s*)(?:"(?:[^"\\]|\\.){0,512}"|'[^']{0,512}'|[^\s&;,'"}]+)/gi,
+        '$1[redacted: secret]',
+    ],
+    // JWT: three base64url segments, the first two starting with an encoded `{"`.
+    // Linear: a start cannot sit inside a base64url run (the lookbehind), and
+    // each segment is capped, so a hostile `-eyJ-eyJ...` costs a fixed amount.
+    [
+        /(?<![A-Za-z0-9_-])eyJ[A-Za-z0-9_-]{5,2048}\.eyJ[A-Za-z0-9_-]{5,2048}\.[A-Za-z0-9_-]{0,2048}/g,
+        '[redacted: token]',
+    ],
+    // An Authorization header's value, including its scheme.
+    [/\b(authorization\s*[:=]\s*)(?:(?:bearer|basic|token)\s+)?[^\s,;'"]+/gi, '$1[redacted: token]'],
+    // A scheme followed by a credential, anywhere.
+    [/\b(bearer|basic)\s+[A-Za-z0-9._~+/=-]{8,}/gi, '$1 [redacted: token]'],
+    // Well-known prefixed token formats.
+    [
+        /\b(?:gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|xox[abprs]-[A-Za-z0-9-]{10,}|(?:AKIA|ASIA)[0-9A-Z]{16}|(?:sk|rk)_(?:live|test)_[A-Za-z0-9]{10,}|AIza[0-9A-Za-z_-]{30,})\b/g,
+        '[redacted: token]',
+    ],
+];
+export function redactPii(text) {
+    let out = text;
+    for (const [pattern, replacement] of PII_PATTERNS)
+        out = out.replace(pattern, replacement);
+    return out;
+}
 /**
  * The same structure with every held secret replaced.
  *

package/dist/errors/tasks.d.ts

@@ -14,3 +14,18 @@ import type { Db } from '../types.js';
  */
 export declare const pruneErrors: Task;
 export declare function pruneErrorsBatch(db: Db, days: number, batch: number): Promise<number>;
+/**
+ * The most open groups one site keeps. `pruneErrors` never deletes an open
+ * group, and the public ingest route lets anyone create one per distinct
+ * message, so without this ceiling `reporting_errors` grows without bound.
+ * Generous for a real app (a few thousand distinct live bugs is already an
+ * incident), tight enough that a flood cannot fill the disk.
+ */
+export declare const MAX_OPEN_ERRORS_PER_SITE = 5000;
+/**
+ * Evicts the least recently seen open groups beyond `MAX_OPEN_ERRORS_PER_SITE`
+ * per site (0007_errors_open_cap.sql). Needs that migration; a host that has
+ * not applied it sees this one task fail and `pruneErrors` unaffected.
+ */
+export declare const pruneOpenErrors: Task;
+export declare function pruneOpenErrorsBatch(db: Db, cap: number, batch: number): Promise<number>;

package/dist/errors/tasks.js

@@ -48,3 +48,50 @@ export async function pruneErrorsBatch(db, days, batch) {
         return Number(row?.n ?? 0);
     });
 }
+/**
+ * The most open groups one site keeps. `pruneErrors` never deletes an open
+ * group, and the public ingest route lets anyone create one per distinct
+ * message, so without this ceiling `reporting_errors` grows without bound.
+ * Generous for a real app (a few thousand distinct live bugs is already an
+ * incident), tight enough that a flood cannot fill the disk.
+ */
+export const MAX_OPEN_ERRORS_PER_SITE = 5000;
+/**
+ * Evicts the least recently seen open groups beyond `MAX_OPEN_ERRORS_PER_SITE`
+ * per site (0007_errors_open_cap.sql). Needs that migration; a host that has
+ * not applied it sees this one task fail and `pruneErrors` unaffected.
+ */
+export const pruneOpenErrors = {
+    name: 'reporting.prune_open_errors',
+    every: HOUR,
+    lease: 10 * 60 * 1000,
+    retry: 15 * 60 * 1000,
+    async run(ctx) {
+        let total = 0;
+        for (let i = 0; i < 20; i += 1) {
+            if (ctx.now().getTime() > ctx.deadline.getTime() - 30_000)
+                break;
+            const n = await pruneOpenErrorsBatch(ctx.db, MAX_OPEN_ERRORS_PER_SITE, 5000);
+            total += n;
+            if (n < 5000)
+                break;
+            if (!(await ctx.renew()))
+                break;
+        }
+        if (total > 0) {
+            ctx.reporting.event({
+                kind: 'reporting.pruned_open_errors',
+                message: `evicted ${total} open error group(s) beyond the per-site cap`,
+                data: { rows: total, cap: MAX_OPEN_ERRORS_PER_SITE },
+            });
+        }
+    },
+};
+export async function pruneOpenErrorsBatch(db, cap, batch) {
+    return db.transaction(async (tx) => {
+        await tx.execute(sql `set local statement_timeout = '30s'`);
+        const result = await tx.execute(sql `select reporting_prune_open_errors(${cap}::int, ${batch}::int) as n`);
+        const [row] = rowsOf(result);
+        return Number(row?.n ?? 0);
+    });
+}