@hotmeshio/long-tail 0.1.4 → 0.1.6

package/docs/iam.md

@@ -0,0 +1,222 @@
+# Identity and Access Management
+
+Every Long Tail workflow executes with identity context. An activity always knows who started the work, whose permissions govern it, and what credentials are available. This is not optional — identity propagates automatically through the durable execution engine.
+
+This document covers the IAM model, how identity flows through the system, and how to configure it for your own workflows.
+
+## Three Identity Dimensions
+
+Each workflow execution carries three pieces of identity:
+
+**Initiator** — the human or cron job that triggered the workflow. Stored as `initiated_by` (a UUID from `lt_users`) on the task record. This never changes, even if the workflow delegates execution to a service account.
+
+**Principal** — the identity the workflow runs as. Usually the initiator, but can be a service account if the workflow is configured with `execute_as` or the request includes an override. The principal determines RBAC permissions and credential access.
+
+**Credentials** — OAuth tokens and API keys available to the principal. Resolved at runtime through a cascade: principal's stored credentials, then the initiating user's credentials, then system environment variables.
+
+## Workflow Types and IAM
+
+Long Tail has three workflow types (see the [Workflows Guide](workflows.md#three-workflow-types) for full details). IAM applies to all three:
+
+**Durable workflows** are the baseline. Every workflow registered with HotMesh is durable: checkpointed to Postgres, restartable after crashes, with full IAM context. If an activity throws, the workflow fails.
+
+**Certified workflows** add the interceptor. The interceptor wraps every execution so that failures escalate to a human reviewer instead of throwing. A certified workflow has an entry in `lt_config_workflows` that defines its escalation chain, invocation roles, and optional `execute_as` service account. It gets never-fail guarantees — when an LLM call returns garbage or an API is down, the workflow pauses and creates a human task rather than dying.
+
+**Pipeline workflows** are compiled deterministic workflows that execute tool calls without an LLM. They inherit the IAM context of the invoking workflow — the principal, credentials, and trace lineage all propagate through the YAML DAG execution.
+
+Any durable workflow can be promoted to certified through the Workflow Registry in the dashboard. Registration adds the interceptor config; de-registration removes it. The workflow code does not change.
+
+## Service Accounts
+
+A service account is a non-human principal — a row in `lt_users` with `account_type = 'bot'`. Service accounts have their own roles, scopes, and stored credentials, identical to human users in every functional respect. The `account_type` field exists for audit segmentation, not authorization.
+
+Service accounts authenticate with API keys prefixed `lt_bot_`. The keys are bcrypt-hashed at rest and validated by the same auth adapter that handles human JWTs. Once authenticated, the request is indistinguishable from a human request.
+
+### When to use a service account
+
+Use `execute_as` when a workflow needs permissions or credentials that differ from the invoking user's. A nightly analytics job might run as a `data-bot` service account that holds a read-only database credential. A CI pipeline might invoke workflows through a `ci-bot` with the `engineer` role.
+
+Two ways to set `execute_as`:
+
+1. **Workflow config** — set `execute_as` in `lt_config_workflows`. Every invocation of that workflow runs as the specified service account.
+2. **Per-request override** — pass `executeAs` in the invocation payload. Requires admin or superadmin role.
+
+In both cases, the original invoker is preserved in `initiated_by`. The audit trail always shows both who asked and who executed.
+
+### Lifecycle
+
+```bash
+# Create a service account (admin-only)
+POST /api/bot-accounts
+{ "name": "data-bot", "description": "Nightly analytics" }
+
+# Assign roles
+POST /api/bot-accounts/:id/roles
+{ "role": "engineer", "type": "member" }
+
+# Generate an API key (returned once, not retrievable later)
+POST /api/bot-accounts/:id/api-keys
+{ "name": "production" }
+# → { "id": "...", "rawKey": "lt_bot_a1b2c3..." }
+
+# Authenticate
+curl -H "Authorization: Bearer lt_bot_a1b2c3..." \
+  http://localhost:3000/api/workflows/mcpQuery/invoke \
+  -d '{"prompt": "Check system health"}'
+```
+
+## Identity Flow
+
+Identity propagates automatically from HTTP request to activity:
+
+```
+HTTP request (JWT or lt_bot_ key)
+  → req.auth.userId
+    → LTEnvelope.lt.userId / lt.executeAs
+      → Interceptor resolves ToolContext from DB (roles, scopes)
+        → Activity interceptor injects principal into argumentMetadata
+          → getToolContext() / getActivityIdentity() in any activity
+```
+
+Inside an activity, call `getActivityIdentity()` to access identity and credentials:
+
+```typescript
+import { getActivityIdentity } from '../services/iam/activity';
+
+export async function fetchData(input: { query: string }) {
+  const identity = getActivityIdentity();
+
+  // Who is executing
+  identity.principal.id;          // UUID
+  identity.principal.type;        // 'user' | 'bot'
+  identity.principal.roles;       // ['engineer', 'admin']
+
+  // Who originally initiated (when execute_as is used)
+  identity.initiatingPrincipal;   // the human who triggered the workflow
+
+  // Credential exchange
+  const token = await identity.getCredential('anthropic');
+}
+```
+
+The `basicEcho` example workflow (`examples/workflows/basic-echo/`) demonstrates all three access patterns: `getActivityIdentity()`, `getToolContext()`, and raw `Durable.activity.getContext()`. Use it to verify IAM propagation in your environment.
+
+## Credential Exchange
+
+Activities resolve credentials at runtime through a cascade:
+
+1. **Executing principal's stored credential** — looks up the service account's or user's encrypted OAuth token in `lt_oauth_tokens`. Expired tokens with refresh tokens are refreshed automatically.
+2. **Initiating principal's credential** — if `execute_as` is active and the service account lacks the credential, falls back to the human invoker's stored token.
+3. **System environment variable** — checks well-known env vars (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, etc.).
+
+If none resolve, `getCredential()` throws `MissingCredentialError`. The credential source is tracked (`user`, `bot`, or `system`) for billing and audit.
+
+```typescript
+// A service account workflow that needs the invoking user's Gmail token
+const identity = getActivityIdentity();
+const gmailToken = await identity.getCredential('google');
+// Cascade: bot's token → human invoker's token → env var → MissingCredentialError
+```
+
+## Ephemeral Credentials
+
+When a workflow needs a credential it does not have — a password, an API key, a one-time token — it escalates to a human. The human provides the value through a form. The challenge: that value must reach the tool that needs it without being logged, stored in workflow state, or exposed to the LLM.
+
+Long Tail solves this with ephemeral credential tokens.
+
+### How it works
+
+1. **Escalation creates a form.** The MCP tool (typically `escalate_and_wait`) includes a `form_schema` with fields marked `format: "password"`. The dashboard renders these as masked password inputs.
+
+2. **User submits the form.** The resolve endpoint (`POST /api/escalations/:id/resolve`) intercepts password fields before signaling the workflow. Each plaintext value is encrypted with AES-256-GCM and stored in `lt_ephemeral_credentials` with a 15-minute TTL. The plaintext is replaced with an opaque token: `eph:v1:password:<uuid>`.
+
+3. **The workflow receives only tokens.** The signal payload contains `eph:v1:...` strings where passwords were. The LLM sees these tokens if it inspects the resolver data — but they are meaningless without exchange.
+
+4. **Exchange happens at dispatch.** When a tool is about to be called, `exchangeTokensInArgs()` walks the argument tree. Every `eph:v1:` string is exchanged atomically: the row's `use_count` increments, the encrypted value is decrypted, and the plaintext is returned. This happens in the activity callback — the latest possible moment before the external API call.
+
+5. **Tokens expire.** After 15 minutes or after exhausting `max_uses`, the row is deleted. No plaintext persists in the database.
+
+### Token lifecycle
+
+| Property | Value |
+|---|---|
+| Format | `eph:v1:<label>:<uuid>` |
+| Encryption | AES-256-GCM |
+| Default TTL | 900 seconds (15 minutes) |
+| Max uses | Unlimited by default, configurable |
+| Exchange tracking | `use_count` incremented atomically on each exchange |
+| Storage | `lt_ephemeral_credentials` table, encrypted at rest |
+
+### Exchange points
+
+Tokens are exchanged in three places, all at the final moment before tool dispatch:
+
+- **MCP tool executor** — dynamic LLM-driven tool calls in mcpQuery workflows
+- **Triage tool executor** — tool calls during mcpTriage remediation
+- **YAML workflow workers** — compiled pipeline tool calls (both DB and MCP server tools)
+
+If a token is expired or exhausted, the opaque string passes through unchanged. The receiving tool will reject it as an unrecognized credential — a safe failure mode.
+
+### MissingCredentialError
+
+When an activity calls `getCredential('anthropic')` and no credential exists in the cascade (principal → initiator → env var), `MissingCredentialError` is thrown. The interceptor catches this and creates a credential-focused escalation with `category: 'missing_credential'`. The escalation form can include a password field so the human provides the credential ephemerally, without it being stored permanently.
+
+## Dashboard
+
+The dashboard surfaces IAM across four pages:
+
+**Workflow Registry** (`/workflows/registry`) — lists all discovered workflows. Certified workflows display a ShieldCheck badge in accent blue; pipeline workflows display a Wand2 icon in purple; durable workflows show the standard Workflow icon. Use ShieldPlus to certify a durable workflow or ShieldOff to de-certify.
+
+**Accounts** (`/admin/users`) — unified management for User Accounts and Service Accounts via tab toggle. Create service accounts, assign roles, generate API keys. The key generation flow displays the raw key once; it cannot be retrieved after dismissal.
+
+**Invoke Workflow** (`/workflows/start`) — all invocable workflows in a single list with visual tier distinction. Certified workflows show the green shield; durable workflows show the standard icon. Both support Start Now and Schedule (cron).
+
+**Connections** (`/credentials`) — each user manages their OAuth provider connections. Status, credential type, and expiry are visible. Users connect or revoke providers here.
+
+## Audit Trail
+
+Every task record carries three audit columns:
+
+| Column | Type | Description |
+|---|---|---|
+| `initiated_by` | UUID | The human or cron that started the workflow chain |
+| `principal_type` | `user` \| `bot` | Type of the executing principal |
+| `executing_as` | string | Service account `external_id` when `execute_as` is active |
+
+These columns enable queries like "show all workflows a service account executed" or "find every workflow initiated by user X but executed by bot Y."
+
+The MCP client also emits debug-level audit logs for every tool invocation:
+
+```
+[lt-mcp:audit] fetchJson on long-tail-http-fetch by user:a1b2c3d4
+[lt-mcp:audit] run_query on long-tail-db by bot:e5f6g7h8
+```
+
+## Security Model
+
+| Credential | Scope | Lifetime | Revocation |
+|---|---|---|---|
+| User JWT | Full API (RBAC-scoped) | 24 hours | Logout / expiry |
+| Bot API key | Full API (RBAC-scoped) | Until revoked | Admin deletes key |
+| Delegation token | Specific scopes | 5 min (max 1 hr) | Expires naturally |
+| OAuth token | Provider scopes | Provider-set | User revokes in Connections |
+
+Service accounts inherit the same RBAC constraints as human users. A service account with the `member` role cannot access admin endpoints. A service account without `engineer` cannot invoke workflows gated by `invocation_roles`.
+
+The `execute_as` override requires admin role. There is no way for a non-admin to impersonate a service account through the API.
+
+## Key Files
+
+| Layer | Path |
+|---|---|
+| Types | `types/tool-context.ts` |
+| Context propagation | `services/iam/context.ts` |
+| Identity resolution | `services/iam/resolve.ts` |
+| Activity identity | `services/iam/activity.ts` |
+| Credential cascade | `services/iam/credentials.ts` |
+| Service account management | `services/iam/bots.ts` |
+| Admin API | `routes/bot-accounts.ts` |
+| Example workflow | `examples/workflows/basic-echo/` |
+| Schema (accounts) | `services/db/schemas/008_bot_accounts.sql` |
+| Schema (audit) | `services/db/schemas/009_audit_trail.sql` |
+| Schema (execute_as) | `services/db/schemas/013_execute_as.sql` |

package/docs/logging.md

@@ -0,0 +1,110 @@
+# Logging
+
+All internal log output in Long Tail flows through a single logger registry. The registry delegates to whatever adapter is registered — Pino, Winston, Bunyan, or any class that implements the `LTLoggerAdapter` interface. Long Tail ships with a ready-made Pino adapter; enable it with `logging: { pino: { level: 'info' } }` in the `start()` config. When no adapter is registered, the registry falls back to `console.*` so log output is never silently dropped, but production deployments should always register a structured adapter. Roughly 15 call sites across the codebase — the main entry point, workers, maintenance routines, event handling, telemetry, database migrations, and the interceptor — use the registry directly. No call site chooses its own transport; all defer to whatever adapter is registered.
+
+## Configuration via start()
+
+The simplest way to configure logging is through the `start()` config:
+
+```typescript
+import { start } from '@hotmeshio/long-tail';
+
+// Built-in Pino adapter
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [ ... ],
+  logging: { pino: { level: 'info' } },
+});
+
+// Custom adapter
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [ ... ],
+  logging: { adapter: new WinstonAdapter() },
+});
+```
+
+The logging adapter is registered first, before any other initialization, so all startup messages flow through it.
+
+## Interface
+
+The adapter contract is defined in `types/logger.ts`:
+
+```typescript
+export type LogLevel = 'debug' | 'info' | 'warn' | 'error' | 'silent';
+
+export interface LTLoggerAdapter {
+  info(msg: string, context?: Record<string, any>): void;
+  warn(msg: string, context?: Record<string, any>): void;
+  error(msg: string, context?: Record<string, any>): void;
+  debug(msg: string, context?: Record<string, any>): void;
+}
+```
+
+Every method accepts a message string and an optional context object. The context is passed as-is to the underlying logger, so structured fields (request IDs, durations, error codes) propagate without serialization on the caller's side.
+
+## Registry
+
+`LTLoggerRegistry` (`services/logger/index.ts`) is a singleton that itself implements `LTLoggerAdapter`. Internal code calls `loggerRegistry.info(...)`, `loggerRegistry.error(...)`, and so on without knowing which adapter, if any, is backing it.
+
+| Method | Purpose |
+|---|---|
+| `register(adapter)` | Set the active adapter. Subsequent log calls delegate to it. |
+| `clear()` | Remove the active adapter and revert to console fallback. |
+| `hasAdapter` | Boolean property indicating whether a custom adapter is registered. |
+
+Registration can happen at any point during the application lifecycle. Calls made before an adapter is registered use the console fallback described below.
+
+## Built-in Pino adapter
+
+Long Tail ships a ready-made Pino adapter in `services/logger/pino.ts`. Its constructor accepts the standard Pino `LoggerOptions` object, so any Pino configuration -- log level, transports, redaction, serializers -- works without modification.
+
+```typescript
+import { loggerRegistry, PinoLoggerAdapter } from '@hotmeshio/long-tail';
+
+loggerRegistry.register(new PinoLoggerAdapter({
+  level: 'info',
+  transport: { target: 'pino-pretty' }, // optional, for development
+}));
+```
+
+### Pino features worth noting
+
+- **JSON output by default.** Each log line is a single JSON object, ready for ingestion by Elasticsearch, Datadog, or any log aggregator that parses structured data.
+- **Sub-millisecond serialization.** Pino's design avoids synchronous string concatenation in the hot path; serialization cost is negligible compared to the I/O it triggers.
+- **First-class TypeScript support.** Type definitions ship with the `pino` package. The `PinoLoggerAdapter` preserves those types without additional wrapping.
+- **OpenTelemetry integration.** The `@opentelemetry/instrumentation-pino` package automatically injects trace and span IDs into every log line when an OTEL SDK is active, correlating logs with distributed traces.
+
+## Custom adapter
+
+Any object that implements `LTLoggerAdapter` can serve as the adapter. Below is a Winston example:
+
+```typescript
+import winston from 'winston';
+import type { LTLoggerAdapter } from '@hotmeshio/long-tail';
+
+class WinstonAdapter implements LTLoggerAdapter {
+  private logger = winston.createLogger({ /* ... */ });
+  info(msg: string, context?: Record<string, any>) { this.logger.info(msg, context); }
+  warn(msg: string, context?: Record<string, any>) { this.logger.warn(msg, context); }
+  error(msg: string, context?: Record<string, any>) { this.logger.error(msg, context); }
+  debug(msg: string, context?: Record<string, any>) { this.logger.debug(msg, context); }
+}
+
+loggerRegistry.register(new WinstonAdapter());
+```
+
+The same pattern applies to Bunyan, log4js, or a bespoke adapter that writes to a database. The only requirement is the four methods.
+
+## Fallback behavior
+
+When no adapter is registered, the registry routes calls to the console:
+
+| Log level | Console method |
+|---|---|
+| `info` | `console.log` |
+| `debug` | `console.log` |
+| `warn` | `console.warn` |
+| `error` | `console.error` |
+
+This ensures that log output is never silently dropped, even in minimal setups where no logging library is installed. Once an adapter is registered via `register()`, the console fallback is bypassed entirely.

package/docs/maintenance.md

@@ -0,0 +1,221 @@
+# DB Maintenance
+
+HotMesh stores execution artifacts in PostgreSQL: stream messages that carry internal signals between activities, transient job rows that track bookkeeping for individual activity invocations, and entity job rows that represent durable workflow instances. Left unmanaged, these tables grow without bound. The maintenance system provides scheduled, rule-based cleanup that runs as a durable cron workflow inside HotMesh itself.
+
+A default configuration ships with Long Tail. It runs nightly at 2 AM, applies four rules in sequence, and requires no setup. The schedule and rules can be replaced at startup or at runtime through the REST API.
+
+## Configuration via start()
+
+Maintenance is enabled by default when you call `start()`. To customize or disable it:
+
+```typescript
+import { start } from '@hotmeshio/long-tail';
+
+// Default: nightly 2 AM cleanup (no config needed)
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [ ... ],
+});
+
+// Custom schedule and rules
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [ ... ],
+  maintenance: {
+    schedule: '0 3 * * *',
+    rules: [
+      { target: 'streams', olderThan: '24 hours', action: 'delete' },
+      { target: 'jobs',    olderThan: '14 days',  action: 'delete', hasEntity: false },
+      { target: 'jobs',    olderThan: '14 days',  action: 'prune',  hasEntity: true },
+      { target: 'jobs',    olderThan: '180 days', action: 'delete', pruned: true },
+    ],
+  },
+});
+
+// Disabled entirely
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [ ... ],
+  maintenance: false,
+});
+```
+
+## What gets cleaned
+
+Four categories of data are subject to maintenance:
+
+| Category | Description |
+|---|---|
+| **Streams** | Redis-style stream messages used internally by HotMesh to coordinate activities. These are pure infrastructure; no user-facing data resides here. |
+| **Transient jobs** | Job rows where the `entity` column is `NULL`. These represent activity executions, signal deliveries, and other bookkeeping that is not tied to a named workflow entity. They serve no purpose after execution completes. |
+| **Entity jobs** | Job rows where `entity` is set. These represent actual workflow instances -- the rows that back `Durable.Client.workflow.search()` calls and execution exports. Deleting them removes the workflow record entirely. |
+| **Pruned jobs** | Entity jobs that have already had their execution artifacts stripped (the `pruned_at` column is not `NULL`). These retain core data but no longer carry execution scaffolding. They can be hard-deleted after a longer retention window. |
+
+## Prune vs. delete
+
+The two actions differ in what they leave behind:
+
+- **Prune** strips execution scaffolding -- activity state, signal payloads, transition metadata -- but preserves the core fields (`jdata`, `udata`, `jmark`, `hmark`). A pruned workflow remains searchable and exportable. Execution exports continue to work.
+- **Delete** removes the row (or stream message) entirely. The data is gone.
+
+The intended lifecycle for an entity job is: execute, then prune after a short retention period to reclaim space, then delete after a longer period once the record is no longer needed.
+
+## Default schedule
+
+The built-in configuration (`modules/maintenance.ts`) runs at 2 AM daily and applies four rules in order:
+
+```typescript
+{
+  schedule: '0 2 * * *',
+  rules: [
+    { target: 'streams', action: 'delete', olderThan: '7 days' },
+    { target: 'jobs', action: 'delete', olderThan: '7 days', hasEntity: false },
+    { target: 'jobs', action: 'prune',  olderThan: '7 days', hasEntity: true },
+    { target: 'jobs', action: 'delete', olderThan: '90 days', pruned: true },
+  ],
+}
+```
+
+| Rule | Effect |
+|---|---|
+| 1. Delete streams older than 7 days | Removes internal message data that is no longer needed for replay or debugging. |
+| 2. Delete transient jobs older than 7 days | Removes activity-level bookkeeping rows that have no associated workflow entity. |
+| 3. Prune entity jobs older than 7 days | Strips execution artifacts from workflow instances while preserving core data. Exports and search remain functional. |
+| 4. Delete pruned jobs older than 90 days | Hard-deletes workflow instances that were pruned at least 90 days ago. This is the final removal. |
+
+Rules execute sequentially. If one fails, the remaining rules still run; failures are logged and do not halt the cycle.
+
+## Configuration types
+
+### `LTMaintenanceRule`
+
+Defined in `types/maintenance.ts`. Each rule describes a single cleanup operation.
+
+| Field | Type | Required | Description |
+|---|---|---|---|
+| `target` | `'streams' \| 'jobs'` | Yes | The resource type to act on. |
+| `action` | `'delete' \| 'prune'` | Yes | Whether to remove entirely or strip execution artifacts. |
+| `olderThan` | `string` | Yes | A PostgreSQL interval expression: `'7 days'`, `'24 hours'`, `'90 days'`. |
+| `hasEntity` | `boolean` | No | When `target` is `'jobs'`: `true` selects entity jobs, `false` selects transient jobs (where `entity IS NULL`). |
+| `pruned` | `boolean` | No | When `true`, only targets jobs where `pruned_at IS NOT NULL` -- jobs that have already been pruned. |
+
+### `LTMaintenanceConfig`
+
+| Field | Type | Description |
+|---|---|---|
+| `schedule` | `string` | A cron expression (`'0 2 * * *'`) or an interval string (`'1 day'`). Passed directly to `Virtual.cron` as the `interval` option. |
+| `rules` | `LTMaintenanceRule[]` | An ordered array of rules. Executed sequentially on each cron tick. |
+
+## Registry
+
+The `maintenanceRegistry` singleton (`services/maintenance/index.ts`) follows the same pattern as `telemetryRegistry` and `eventRegistry`:
+
+| Method | Purpose |
+|---|---|
+| `register(config)` | Store a maintenance configuration. Replaces any previously registered config. Call before `connect()`. |
+| `connect()` | Start the `Virtual.cron` workflow that fires on the configured schedule. Idempotent -- given the same internal cron ID, duplicate calls do not create duplicate schedules. |
+| `disconnect()` | Cancel the running cron by calling `Virtual.interrupt`. Safe to call if no cron is running. |
+| `clear()` | Remove the config and reset internal state. Used in tests. |
+| `hasConfig` | Boolean property. `true` when a config has been registered. |
+| `config` | Returns the current `LTMaintenanceConfig` or `null`. |
+
+## Programmatic registration
+
+For advanced use cases, you can register maintenance programmatically instead of through `start()`. The `start()` function handles this automatically when a `maintenance` config is provided (or uses the default when omitted):
+
+```typescript
+import { maintenanceRegistry, defaultMaintenanceConfig } from '@hotmeshio/long-tail';
+
+maintenanceRegistry.register(defaultMaintenanceConfig);
+```
+
+To use a custom config instead, register it in place of the default:
+
+```typescript
+import { maintenanceRegistry } from '@hotmeshio/long-tail';
+
+maintenanceRegistry.register({
+  schedule: '0 3 * * 0',  // Sundays at 3 AM
+  rules: [
+    { target: 'streams', action: 'delete', olderThan: '14 days' },
+    { target: 'jobs', action: 'delete', olderThan: '14 days', hasEntity: false },
+    { target: 'jobs', action: 'prune', olderThan: '30 days', hasEntity: true },
+    { target: 'jobs', action: 'delete', olderThan: '180 days', pruned: true },
+  ],
+});
+```
+
+The registry accepts exactly one config at a time. Calling `register()` again overwrites the previous config. The cron does not start until `connect()` is called — which happens inside `start()` when `hasConfig` is `true`.
+
+## Runtime API
+
+The REST API allows administrators to replace the maintenance config while the server is running.
+
+### `GET /api/config/maintenance`
+
+Returns the current configuration and whether a cron is active.
+
+```json
+{
+  "config": {
+    "schedule": "0 2 * * *",
+    "rules": [ ... ]
+  },
+  "active": true
+}
+```
+
+### `PUT /api/config/maintenance`
+
+Admin-only (requires `requireAdmin` middleware). Accepts a full replacement config in the request body:
+
+```json
+{
+  "schedule": "0 4 * * *",
+  "rules": [
+    { "target": "streams", "action": "delete", "olderThan": "3 days" },
+    { "target": "jobs", "action": "delete", "olderThan": "3 days", "hasEntity": false },
+    { "target": "jobs", "action": "prune", "olderThan": "3 days", "hasEntity": true },
+    { "target": "jobs", "action": "delete", "olderThan": "60 days", "pruned": true }
+  ]
+}
+```
+
+The endpoint performs three operations in sequence:
+
+1. **Disconnect** -- calls `maintenanceRegistry.disconnect()` to cancel the running cron via `Virtual.interrupt`.
+2. **Register** -- calls `maintenanceRegistry.register()` with the new config.
+3. **Connect** -- calls `maintenanceRegistry.connect()` to start a new cron with the updated schedule and rules.
+
+Returns the new config and `{ "restarted": true }` on success. Returns `400` if `schedule` or `rules` is missing, and `500` if any step fails.
+
+## Custom schedule example
+
+A production deployment that processes high volumes might want aggressive short-term cleanup with a longer archive window:
+
+```typescript
+import { maintenanceRegistry } from '@hotmeshio/long-tail';
+
+maintenanceRegistry.register({
+  schedule: '0 */6 * * *',  // every 6 hours
+  rules: [
+    { target: 'streams', action: 'delete', olderThan: '24 hours' },
+    { target: 'jobs', action: 'delete', olderThan: '48 hours', hasEntity: false },
+    { target: 'jobs', action: 'prune', olderThan: '3 days', hasEntity: true },
+    { target: 'jobs', action: 'delete', olderThan: '365 days', pruned: true },
+  ],
+});
+```
+
+This cleans streams and transient jobs within two days, prunes entity jobs after three days (preserving export capability), and retains pruned records for a full year before final deletion.
+
+## How it works internally
+
+The maintenance system runs on top of HotMesh's `Virtual.cron`, the same primitive that powers durable recurring workflows throughout the platform.
+
+1. **Registration.** `maintenanceRegistry.register(config)` stores the config in memory. No I/O occurs.
+2. **Connection.** `maintenanceRegistry.connect()` calls `Virtual.cron()` with a fixed topic (`lt.maintenance.prune`) and a fixed cron ID (`lt-maintenance-nightly`). HotMesh persists the cron schedule in PostgreSQL, making it durable across restarts. If a cron with that ID already exists, it is replaced.
+3. **Execution.** On each tick of the schedule, the cron callback iterates the rules array. For each rule, it translates the rule's fields into the appropriate `dbaService.prune()` parameters and executes the call. Rules run sequentially; a failure in one rule is logged and does not prevent subsequent rules from running.
+4. **Disconnection.** `maintenanceRegistry.disconnect()` calls `Virtual.interrupt()` with the same topic and cron ID, which cancels the recurring schedule. This is called during graceful shutdown and before reconfiguration via the REST API.
+
+Because the cron is itself a durable workflow, the schedule survives process restarts. If the server goes down at 2 AM and comes back at 2:05 AM, the missed tick executes on reconnection.