@ontrails/core 1.0.0-beta.15 → 1.0.0-beta.17

package/CHANGELOG.md

@@ -1,5 +1,74 @@
 # @ontrails/core
 
+## 1.0.0-beta.17
+
+### Patch Changes
+
+- 3dc8254: Fix README TypeScript snippets so the expanded documentation snippet gate can verify them.
+
+## 1.0.0-beta.16
+
+### Major Changes
+
+- 331e3a9: Relocate the topo-store public API from `@ontrails/core` to `@ontrails/topographer` per ADR-0042. Generic `trails-db` helpers (`openReadTrailsDb`, `openWriteTrailsDb`, `ensureSubsystemSchema`, `deriveTrailsDbPath`, `deriveTrailsDir`) stay in core because tracing and other subsystems share them.
+
+  Breaking pre-1.0 beta change. Update consumer imports:
+
+  ```diff
+  - import { topoStore, createTopoStore, createMockTopoStore, createTopoSnapshot, listTopoSnapshots, pinTopoSnapshot, unpinTopoSnapshot, createStoredTopoSnapshot, getStoredTopoExport, countTopoSnapshots, countPinnedSnapshots, countPrunableSnapshots, pruneUnpinnedSnapshots } from '@ontrails/core';
+  + import { topoStore, createTopoStore, createMockTopoStore, createTopoSnapshot, listTopoSnapshots, pinTopoSnapshot, unpinTopoSnapshot } from '@ontrails/topographer';
+  + import { createStoredTopoSnapshot, getStoredTopoExport, countTopoSnapshots, countPinnedSnapshots, countPrunableSnapshots, pruneUnpinnedSnapshots } from '@ontrails/topographer/backend-support';
+  ```
+
+  The same root move applies to types `ReadOnlyTopoStore`, `MockTopoStoreSeed`, `TopoSnapshot`, `TopoStoreRef`, `TopoStoreExportRecord`, `TopoStoreResourceRecord`, `TopoStoreTrailRecord`, `TopoStoreTrailDetailRecord`, `CreateTopoSnapshotInput`, and `ListTopoSnapshotsOptions`. The direct DB helper type `StoredTopoExport` moves to `@ontrails/topographer/backend-support`.
+
+  Core newly exports `activationSourceKey`, `projectActivationSourceDeclaration`, `activationSourceDeclarationSignature`, and the `ActivationSourceProjection` type — these were already used internally and are now part of the public surface so `@ontrails/topographer` (the only consumer that needs them) can import them through normal package channels.
+
+- 10eae9a: Migrate the Trails workspace to the documented `.trails/` layout: committed `.lock` files at the workspace root, ignored `cache/` for rebuildable derived data, ignored `state/` for mutable runtime state, and `.trails/config.local.{ts,js}` for local overrides. The default SQLite path is now `.trails/state/trails.db`. Workspace bootstrap creates only `cache/` and `state/` — the legacy `dev/` and `generated/` subdirectories are no longer created. Dev reset cleans both the new `.trails/state/` paths and legacy `.trails/trails.db*` and `.trails/dev/tracing.db*` paths for one cycle. Scaffold and workspace gitignores reflect the new layout.
+
+  Workspace bootstrap is now owned by a single canonical source in `@ontrails/core`. The package exposes `ensureTrailsWorkspace()`, `WORKSPACE_GITIGNORE_CONTENT`, and `WORKSPACE_GITIGNORE_LINES`. `@ontrails/config` no longer exports its own `ensureWorkspace` (consumers should import from `@ontrails/core`). `trails create` now writes `.trails/.gitignore` during scaffolding so a fresh-scaffolded project's initial commit includes the workspace gitignore (resolves TRL-703).
+
+### Minor Changes
+
+- bcdc484: Add an explicit `unmockable: { reason }` resource marker and have testing auto-mock resolution skip intentionally unmockable resources.
+- 4b8d13b: **BREAKING:** Complete the `trailhead` to `surface` public API cutover in core.
+
+  - `TraceRecord.trailhead` is now `TraceRecord.surface`.
+  - `SURFACE_KEY` now uses the `__trails_surface` extension key value, and the deprecated `TRAILHEAD_KEY` alias is removed.
+  - Deprecated `transport*` surface-error aliases are removed; import the existing `surface*` names instead.
+  - `isVisibleToTrailheads` is renamed to `isVisibleToSurfaces`.
+
+  See `docs/migration/trailhead-to-surface.md` for the full migration map.
+
+- 112b9f2: Add `dryRun` to `TrailContext` and wire `--dry-run`. `TrailContext` gains an optional `dryRun?: boolean` field, defaulted to `false` in `createTrailContext`. `ExecuteTrailOptions` carries `dryRun?: boolean` through `applyContextOverrides` so `executeTrail(t, input, { dryRun: true })` produces `ctx.dryRun === true`. The CLI's existing `dryRunPreset` (auto-derived for `intent: 'write' | 'destroy'` trails) now flows through `META_FLAG_CANDIDATES` and reaches the executor via `runTrailOnce`, so `trails run booking.cancel … --dry-run` lands `ctx.dryRun === true` on the trail's blaze. Trails that don't read the field are unchanged. Read-intent trails don't get `--dry-run` exposed.
+- 893025e: Add `--permit '<json>'` to inject an inline permit on `trails run`. New `permitPreset()` exposes a `--permit` string flag that the CLI build parses and validates against the `BasePermit` shape (`{ id: string, scopes: string[] }`) using a small Zod schema. Valid permits flow through `ExecuteTrailOptions.permit` → `applyContextOverrides` → `ctx.permit` so existing `enforcePermitRequirement` behavior just sees a populated permit. Invalid JSON or schema mismatch surface as `Result.err(ValidationError)` (exit code 1) before the trail runs, avoiding spurious `PermitError` results from malformed input. The flag is global, never routed into trail input (added to `META_FLAG_CANDIDATES`), and overlays only when defined.
+
+  Topographer now projects permit requirements into surface-map entries and classifies permit-tightening diffs as breaking when new scopes are required.
+
+- eec5e9d: Default `ctx.logger` to a structured stdout console sink when `topo()` is called without an `observe:` option. Apps now get observability for free with zero configuration, per ADR-0041. Explicit `observe:` values (including `combine()` with no sinks, an explicit `Logger`, or an explicit `{ log }` config) are preserved untouched — the default is only injected when no `observe:` is supplied.
+- ebd4434: Phase 7 starts: typed layers with optional object-shaped surface input. `Layer` gains an optional `input?: LayerInputSchema` field for surface projection (TRL-473/474 will project it onto CLI/MCP/HTTP). `executeTrail({ layers })` remains the canonical per-call wrapper option. Layers without `input` schemas stay surface-invisible and cover runtime-only concerns such as tenant guards, rate limiting, circuit breaking, and custom audit logging.
+- 863d473: Add three attachment scopes for typed layers — trail, surface, topo — with composition order **topo → surface → trail → blaze**. `TrailSpec` and `Trail` gain `layers?: readonly Layer[]` (default `[]`). `topo()` accepts `{ layers: [...] }` as the third options argument; the topo carries those layers and they reach the executor via `ExecuteTrailOptions.topoLayers`. The CLI's `surface()`/`createProgram()`/`deriveCliCommands` already supports a `layers` option; that now flows through `runTrailOnce` as `surfaceLayers`. The executor builds the layer chain `[...topoLayers, ...surfaceLayers, ...trail.layers, ...options.layers]` so topo wraps surface wraps trail wraps blaze (verified by composition-order tests at every level). Survey's `TrailDetailReport` adds `composedLayers: { topo, surface, trail }` so agents can introspect the layer chain per trail. Backward-compatible: every new field is optional with a non-undefined default; existing call sites are unchanged.
+- 344f2f7: Project typed-layer `input` schemas onto CLI flags. Each effective layer (topo + surface + trail composition order) with a non-undefined `input` schema gets its fields auto-derived into `--flag` options on every command it attaches to. Parsed values route to the layer at runtime via `ctx.extensions[LAYER_INPUTS_KEY][layer.name]` — `Layer.wrap` is unchanged. Collision rule: if a layer field name collides with a trail input field, another layer's projected name, or a CLI meta flag, the layer's flag is renamed to `--<layer-name>-<original-flag-name>` and a one-line warning emits to stderr (`[trails] ...`). Renames are deterministic across builds. New `LAYER_INPUTS_KEY` (exported from `@ontrails/core`) reserves the `ctx.extensions` slot.
+- 26f9ffd: Project typed-layer `input` schemas onto MCP and HTTP surfaces. Closes Phase 7. Lifts `collectAttachedTypedLayers` and `projectLayerFieldName` (collision-rename rule) into `@ontrails/core/internal/layer-projection` so all three surfaces share one source of truth. The CLI surface refactors to consume the lifted helpers (no behavior change). MCP merges layer fields into each tool's `inputSchema` and partitions inbound args at invocation time. HTTP merges layer fields into the route's request schema (query for reads, body for writes) and exposes new optional `HttpRouteDefinition.inputSchema` + `layerInputProjections` for surface adapters / OpenAPI generators. Collision rule matches TRL-473's: deterministic rename to a layer-prefixed camelCase name with the original captured in the routing table. Side fix: MCP and HTTP handlers now forward `topoLayers: graph.layers` + `surfaceLayers: layers` so topo-scope layers actually compose at runtime (previously the handlers used the deprecated `layers` alias and never read `graph.layers`).
+- 22c6c06: Accept ADR-0041 Unified Observability and ship the first activation and
+  observability primitives it depends on: activation trace records, topo-level
+  observe configuration, webhook activation materialization, signal/webhook
+  warden coaching, the `@ontrails/observe` package, sink composition, and
+  zero-dependency observe sinks.
+
+### Patch Changes
+
+- 73622ae: Thread `ResourceSpec.config` through the built-in auth resource. Resource config schemas that accept `undefined` now receive their parsed default when config values are omitted, and `authResource` can materialize the no-op or JWT adapter from typed config while preserving existing mock and override paths.
+- 6300f70: Refresh source comments and test labels for retired connector terminology as adapter guardrails become strict.
+- d172013: Preserve specialized TrailsError identity and retry-exhaustion metadata when
+  serializing and deserializing framework errors.
+- c3fc5c3: Move previously root-exported helper contracts out of `src/internal/*` to stable core module homes, document their public boundary, and guard the public barrel against future internal re-exports.
+- 20d7a5c: Enforce the shared safe error projection policy for public error bodies, diagnostics, serialized payloads, and CLI stderr.
+- be5fb46: Publish registry-driven error taxonomy documentation and checks from the core error registry.
+- e898cc4: Add repo-level Knip dead-code detection and remove stale internal exports and unused package dependencies surfaced by the new check.
+- 3395234: Move store adapter-binding helpers to `@ontrails/store/adapter-support` and topographer direct database/admin helpers to `@ontrails/topographer/backend-support`, keeping root exports focused on contract-level APIs.
+- 4399fdb: Renamed `@ontrails/schema` to `@ontrails/topographer`. Mechanical rename only — no API changes. Update import sites from `@ontrails/schema` to `@ontrails/topographer`. See ADR-0042 for the durable graph substrate doctrine.
+
 ## 1.0.0-beta.15
 
 ### Minor Changes

package/README.md

@@ -44,7 +44,9 @@ const onboard = trail('entity.onboard', {
 | `trail(id, spec)` | Define a unit of work with typed input and `Result` output; use `crosses` for composition |
 | `signal(id, spec)` | Define a server-originated notification with a typed data schema |
 | `resource(id, spec)` | Define an infrastructure dependency with `create`, `dispose`, and optional `mock` |
-| `topo(name, ...modules)` | Collect trail modules into a queryable topology |
+| `drainResources(resources, ctx, configValues?)` | Evict and dispose cached resource singletons for surface/test shutdown |
+| `blobRefSchema` / `createBlobRef(...)` | Declare and create binary output references with a shared descriptor contract |
+| `topo(name, ...modules, options?)` | Collect trail modules into a queryable topology with optional `observe:` sinks |
 | `deriveTrail(contour, operation, spec)` | Derive CRUD-shaped trail contracts from a contour on the `@ontrails/core/trails` subpath |
 | `validateTopo(topo)` | Structural validation: cross targets exist, no cycles, examples parse, output schemas present |
 
@@ -57,11 +59,11 @@ const onboard = trail('entity.onboard', {
 
 ```typescript
 // executeTrail — surfaces use this directly
-const result = await executeTrail(greet, { name: 'Alice' });
+const surfaceResult = await executeTrail(greet, { name: 'Alice' });
 
 // run — headless execution by trail ID
-const result = await run(graph, 'greet', { name: 'Alice' });
-if (result.isOk()) console.log(result.value);
+const runResult = await run(graph, 'greet', { name: 'Alice' });
+if (runResult.isOk()) console.log(runResult.value);
 ```
 
 ### Topo accessors
@@ -113,22 +115,35 @@ result.unwrapOr(fallback);   // Value or fallback
 
 ### Error taxonomy
 
-15 error classes across 10 categories. Each maps deterministically to exit codes, HTTP status, and JSON-RPC codes on every surface.
+The current taxonomy is generated from the `errorClasses` owner registry and
+category code maps in `@ontrails/core`.
 
-| Category | Classes | HTTP | Retryable |
-| --- | --- | --- | --- |
-| `validation` | `ValidationError`, `AmbiguousError` | 400 | No |
-| `not_found` | `NotFoundError` | 404 | No |
-| `conflict` | `AlreadyExistsError`, `ConflictError` | 409 | No |
-| `permission` | `PermissionError` | 403 | No |
-| `timeout` | `TimeoutError` | 504 | Yes |
-| `rate_limit` | `RateLimitError` | 429 | Yes |
-| `network` | `NetworkError` | 502 | Yes |
-| `internal` | `InternalError`, `DerivationError`, `AssertionError` | 500 | No |
-| `auth` | `AuthError` | 401 | No |
-| `cancelled` | `CancelledError` | 499 | No |
+<!-- error-taxonomy:start -->
+<!-- GENERATED: run `bun run error-taxonomy:sync`; check with `bun run error-taxonomy:check`. Variant: category. -->
 
-`RetryExhaustedError` wraps another `TrailsError`, inherits the wrapped error's category for surface mappings, and always reports `retryable: false`.
+| Category | CLI Exit | HTTP | JSON-RPC | Retryable | Fixed Classes |
+| --- | --- | --- | --- | --- | --- |
+| `validation` | 1 | 400 | -32602 | No | `ValidationError`, `AmbiguousError` |
+| `not_found` | 2 | 404 | -32601 | No | `NotFoundError` |
+| `conflict` | 3 | 409 | -32603 | No | `AlreadyExistsError`, `ConflictError` |
+| `permission` | 4 | 403 | -32600 | No | `PermissionError`, `PermitError` |
+| `timeout` | 5 | 504 | -32603 | Yes | `TimeoutError` |
+| `rate_limit` | 6 | 429 | -32603 | Yes | `RateLimitError` |
+| `network` | 7 | 502 | -32603 | Yes | `NetworkError` |
+| `internal` | 8 | 500 | -32603 | No | `AssertionError`, `InternalError`, `DerivationError`, `RecoverableCompletionError` |
+| `auth` | 9 | 401 | -32600 | No | `AuthError` |
+| `cancelled` | 130 | 499 | -32603 | No | `CancelledError` |
+
+Dynamic classes:
+
+- `RetryExhaustedError` inherits category and surface codes from its wrapped `TrailsError`; retryable is always No.
+<!-- error-taxonomy:end -->
+
+Public surface projections redact sensitive substrings before exposing a
+non-internal `TrailsError` message. Internal-category `TrailsError` instances and
+unknown native errors project with the generic message `Internal server error`;
+diagnostics and serialized payloads keep their useful structure while redacting
+messages, context, and stack strings.
 
 The developer returns `Result.err(new NotFoundError(...))`. The framework maps it to the right code on every surface.
 
@@ -140,14 +155,44 @@ The developer returns `Result.err(new NotFoundError(...))`. The framework maps i
 - **Resilience** -- `retry`, `withTimeout`, `shouldRetry`, `deriveBackoffDelay`
 - **Serialization** -- `serializeError`, `deserializeError`
 - **Branded types** -- `uuid`, `email`, `nonEmptyString`, `positiveInt`
-- **Layers** -- cross-cutting layers via `composeLayers`
+- **Execution layers** -- low-level pipeline wrappers via `composeLayers`
 - **Guards and collections** -- `isDefined`, `chunk`, `dedupe`, `groupBy`, `sortBy`
 - **Patterns** (`@ontrails/core/patterns`) -- reusable Zod schemas for pagination, bulk ops, timestamps, sorting
 - **Trail factories** (`@ontrails/core/trails`) -- derive CRUD-shaped trail contracts from contours without re-authoring IDs, schemas, examples, or intents
 - **Redaction** (`@ontrails/core/redaction`) -- strip sensitive data before logging
 
+### Public helper boundaries
+
+The root package also exposes a few low-level contracts that other framework
+packages build on:
+
+- **Intrinsic tracing** -- `TraceRecord`, `TraceSink`, `TraceContext`, and the
+  sink registry helpers are the core-owned execution record shape shared by
+  `@ontrails/observe`, `@ontrails/tracing`, and adapters.
+- **Trails DB** -- `deriveTrailsDbPath`, `deriveTrailsDir`,
+  `ensureSubsystemSchema`, `openReadTrailsDb`, and `openWriteTrailsDb` are the
+  generic database primitive used by framework subsystems.
+- **Surface projection helpers** -- safe error projection, layer field
+  projection, cross-batch validation, late-bound signal references, and Zod
+  default-wrapper stripping are stable root exports for first-party surfaces,
+  store helpers, and tests.
+
 See the [API Reference](../../docs/api-reference.md) for the full list.
 
+## Migration: topo-store moved to `@ontrails/topographer`
+
+Per [ADR-0042](../../docs/adr/0042-core-topographer-boundary-doctrine.md), the topo-store public API previously exported from `@ontrails/core` now lives in `@ontrails/topographer`. Generic `trails-db` helpers (`openReadTrailsDb`, `openWriteTrailsDb`, `ensureSubsystemSchema`, `deriveTrailsDbPath`, `deriveTrailsDir`) stay in core because tracing and other subsystems share them.
+
+Update consumer imports:
+
+```diff
+- import { topoStore, createTopoStore, createMockTopoStore, createTopoSnapshot, listTopoSnapshots, pinTopoSnapshot, unpinTopoSnapshot, createStoredTopoSnapshot, getStoredTopoExport, countTopoSnapshots, countPinnedSnapshots, countPrunableSnapshots, pruneUnpinnedSnapshots } from '@ontrails/core';
++ import { topoStore, createTopoStore, createMockTopoStore, createTopoSnapshot, listTopoSnapshots, pinTopoSnapshot, unpinTopoSnapshot } from '@ontrails/topographer';
++ import { createStoredTopoSnapshot, getStoredTopoExport, countTopoSnapshots, countPinnedSnapshots, countPrunableSnapshots, pruneUnpinnedSnapshots } from '@ontrails/topographer/backend-support';
+```
+
+Types `ReadOnlyTopoStore`, `MockTopoStoreSeed`, `TopoSnapshot`, `TopoStoreRef`, `TopoStoreExportRecord`, `TopoStoreResourceRecord`, `TopoStoreTrailRecord`, `TopoStoreTrailDetailRecord`, `CreateTopoSnapshotInput`, and `ListTopoSnapshotsOptions` move to `@ontrails/topographer`. `StoredTopoExport` moves to `@ontrails/topographer/backend-support`.
+
 ## Installation
 
 ```bash

package/package.json

@@ -1,16 +1,17 @@
 {
   "name": "@ontrails/core",
-  "version": "1.0.0-beta.15",
+  "version": "1.0.0-beta.17",
+  "files": [
+    "src/**/*.ts",
+    "!src/**/__tests__/**",
+    "!src/**/*.test.ts",
+    "!src/**/*.test-d.ts",
+    "README.md",
+    "CHANGELOG.md"
+  ],
   "type": "module",
   "exports": {
     ".": "./src/index.ts",
-    "./internal/cross-batch": "./src/internal/cross-batch.ts",
-    "./internal/signal-ref": "./src/internal/signal-ref.ts",
-    "./internal/topo-snapshots": "./src/internal/topo-snapshots.ts",
-    "./internal/topo-store": "./src/internal/topo-store.ts",
-    "./internal/tracing": "./src/internal/tracing.ts",
-    "./internal/trails-db": "./src/internal/trails-db.ts",
-    "./internal/zod-wrappers": "./src/internal/zod-wrappers.ts",
     "./patterns": "./src/patterns/index.ts",
     "./redaction": "./src/redaction/index.ts",
     "./store": "./src/store/index.ts",

package/src/activation-provenance.ts

@@ -0,0 +1,111 @@
+import type {
+  ActivationSourceKind,
+  ActivationSourceMeta,
+} from './activation-source.js';
+
+export const ACTIVATION_PROVENANCE_KEY =
+  '__trails_activation_provenance' as const;
+
+export interface ActivationProvenanceSource {
+  readonly cron?: string | undefined;
+  readonly id: string;
+  readonly kind: ActivationSourceKind;
+  readonly meta?: ActivationSourceMeta | undefined;
+  readonly producerTrailId?: string | undefined;
+  readonly timezone?: string | undefined;
+}
+
+export interface ActivationProvenance {
+  readonly fireId: string;
+  readonly parentFireId?: string | undefined;
+  readonly rootFireId: string;
+  readonly source: ActivationProvenanceSource;
+}
+
+export interface ActivationProvenanceCarrier {
+  readonly activation?: ActivationProvenance | undefined;
+  readonly extensions?: Readonly<Record<string, unknown>> | undefined;
+}
+
+const isObjectRecord = (value: unknown): value is Record<string, unknown> =>
+  typeof value === 'object' && value !== null && !Array.isArray(value);
+
+const optionalString = (value: unknown): boolean =>
+  value === undefined || typeof value === 'string';
+
+const isActivationProvenanceSource = (
+  value: unknown
+): value is ActivationProvenanceSource =>
+  isObjectRecord(value) &&
+  typeof value['id'] === 'string' &&
+  typeof value['kind'] === 'string' &&
+  optionalString(value['cron']) &&
+  optionalString(value['producerTrailId']) &&
+  optionalString(value['timezone']) &&
+  (value['meta'] === undefined || isObjectRecord(value['meta']));
+
+const isActivationProvenance = (
+  value: unknown
+): value is ActivationProvenance =>
+  isObjectRecord(value) &&
+  typeof value['fireId'] === 'string' &&
+  optionalString(value['parentFireId']) &&
+  typeof value['rootFireId'] === 'string' &&
+  isActivationProvenanceSource(value['source']);
+
+export const getActivationProvenance = (
+  ctx: ActivationProvenanceCarrier | undefined
+): ActivationProvenance | undefined => {
+  if (isActivationProvenance(ctx?.activation)) {
+    return ctx.activation;
+  }
+  const fromExtensions = ctx?.extensions?.[ACTIVATION_PROVENANCE_KEY];
+  return isActivationProvenance(fromExtensions) ? fromExtensions : undefined;
+};
+
+export const withActivationProvenance = <
+  TCtx extends {
+    readonly extensions?: Readonly<Record<string, unknown>> | undefined;
+  },
+>(
+  ctx: TCtx,
+  activation: ActivationProvenance
+): TCtx & { readonly activation: ActivationProvenance } => ({
+  ...ctx,
+  activation,
+  extensions: {
+    ...ctx.extensions,
+    [ACTIVATION_PROVENANCE_KEY]: activation,
+  },
+});
+
+export const buildActivationProvenanceTraceAttrs = (
+  activation: ActivationProvenance | undefined
+): Readonly<Record<string, unknown>> => {
+  if (activation === undefined) {
+    return {};
+  }
+
+  const attrs: Record<string, unknown> = {
+    'trails.activation.fire_id': activation.fireId,
+    'trails.activation.root_fire_id': activation.rootFireId,
+    'trails.activation.source.id': activation.source.id,
+    'trails.activation.source.kind': activation.source.kind,
+  };
+
+  if (activation.parentFireId !== undefined) {
+    attrs['trails.activation.parent_fire_id'] = activation.parentFireId;
+  }
+  if (activation.source.producerTrailId !== undefined) {
+    attrs['trails.activation.source.producer_trail.id'] =
+      activation.source.producerTrailId;
+  }
+  if (activation.source.cron !== undefined) {
+    attrs['trails.activation.source.cron'] = activation.source.cron;
+  }
+  if (activation.source.timezone !== undefined) {
+    attrs['trails.activation.source.timezone'] = activation.source.timezone;
+  }
+
+  return attrs;
+};