autotel-eventcatalog 1.0.1 → 2.0.1

package/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 2.0.0
+
+### Minor Changes
+
+- 9fbbc3a: Close the loop from "code declares the event contract" through to "catalog reflects the runtime" — schemas declared at the `track()` call site flow through telemetry into the catalog generator and drift detector with no inference guesswork.
+
+  ### `autotel`
+  - **New: `defineEvent(name, schema, options?)`.** Returns a `DefinedEvent` that validates the payload at runtime (via the schema's `safeParse`) and carries the JSON Schema and a stable SHA-256 schema hash through `track()` as part of the `EventTrackingOptions`. Designed for Zod (`{ toJsonSchema: (s) => z.toJSONSchema(s) }`) but accepts any schema with a `safeParse` method. Imported from `autotel`.
+  - **New `schema?: EventSchemaMetadata` field** on `EventTrackingOptions`. The `EventQueue` carries it onto the `EventPayload` so any contract-aware subscriber (`ArchitectureSnapshotSubscriber`, custom subscribers) sees the declared schema verbatim. Optional and backwards-compatible — bare `track()` calls continue to work.
+  - **PII redaction now only applies to string values.** The default sensitive-key patterns (`/token/i`, `/auth/i`, …) used to overwrite _any_ matching value — including numbers and booleans — with the literal string `"[REDACTED]"`. That broke type stability for fields like `promptTokens` / `completionTokens` (LLM usage counters) and gave nothing in return: secrets in user code are overwhelmingly strings. Numeric and boolean attributes now pass through untouched. Same change applied to `AttributeRedactingProcessor`. Existing tests that asserted booleans got redacted have been updated to reflect the new (correct) behaviour.
+
+  ### `autotel-subscribers`
+  - **`ArchitectureSnapshotSubscriber` records `fieldStats` for every observed event.** For each dotted field path it tracks the runtime types it saw (`string`, `number`, `object`, …) and up to 20 primitive sample values, merging across observations. The new `FieldStats` type is added to `EventObservation`. Existing snapshots stay valid — `fieldStats` is optional.
+  - **Captures declared schemas from `defineEvent`.** When a `track()` call originated from `defineEvent`, the subscriber stores the declared `{ source: 'zod', jsonSchema, hash }` on the observation as `EventObservation.schema?`. Snapshots that go through bare `track()` are unchanged.
+  - **Captures consumer-service attribution.** A new optional `_autotel.consumers: string[]` convention on the event attributes is read into `EventObservation.consumers?`, so the snapshot now describes consumer relationships in addition to producer / channel.
+
+  ### `autotel-eventcatalog`
+  - **New `generate` command** — scaffolds services, events, channels, and producer/consumer/channel-routing edges from a snapshot. Skip-if-exists: catalog files that already exist are left completely untouched. When the snapshot's `EventObservation.schema?.jsonSchema` is present (declared at the `track()` call site), it is written verbatim to the event's `schema.json`. Otherwise the schema is inferred from runtime `fieldStats` as a fallback — captured in the operations log as `schemaSource: 'declared' | 'inferred'`. CLI flags: `--snapshot`, `--catalog`, `--dry-run`, `--edges-only`, `--version`, `--summary-output`. Versioned summary envelope (`schemas/generate-summary-v0.1.0.json`) pinned by a contract test.
+  - **Type drift and value drift** — `diffCatalogAgainstSnapshot` now consumes `fieldStats` to detect runtime-vs-declared mismatches. Drift detector handles the JSON Schema `integer` vs JS `number` impedance mismatch deliberately: declared `integer` accepts observed `number` at the type level, but sample values are checked against `Number.isInteger` — so a runtime `1.5` against an `integer` declaration still flags. No false positives on integer fields, no swept-under-the-rug genuine signal. New `drift-report-v0.2.0.json` and `drift-summary-v0.2.0.json` schemas pin the richer wire format; v0.1.0 envelopes are still emitted for backwards-compatible consumers.
+  - **`SnapshotDiff` interop renderer** — `toSnapshotDiffFromReport(report)` and `toSnapshotDiffFromDelta(delta)` produce EventCatalog's own `SnapshotDiff` shape (with `ResourceChange[]` and `RelationshipChange[]`), so drift findings can flow into upstream catalog tooling that already understands that format. Exposed as the new `eventcatalog-snapshot-diff` renderer.
+  - **Catalog state now read via `@eventcatalog/sdk`** instead of a bespoke filesystem walker. `CatalogEvent`, `CatalogService` and `CatalogChannel` extend the SDK's `Event`, `Service` and `Channel` types directly, so any field the SDK adds in future is picked up without changes here. The package gains `@eventcatalog/sdk` as a runtime dependency.
+  - **Workaround for an upstream SDK bug** — `addEventToChannel` in `@eventcatalog/sdk@2.21.2` corrupts catalog layout (turns `index.mdx` into a directory) because of a string-vs-regex bug in path splitting. `generate` sets the channel pointer directly on the event's frontmatter when calling `writeEvent`, sidestepping the bad code path. The fix is filed upstream as [event-catalog/eventcatalog#2567](https://github.com/event-catalog/eventcatalog/pull/2567); the workaround can be removed once that ships.
+
+  ### What this means in practice
+
+  The reference app (`apps/example-eventcatalog`) has been migrated to `defineEvent` for all five domain events. Running `pnpm services:snapshot && pnpm catalog:drift` against the resulting catalog now prints `No drift detected. Catalog and runtime agree.` That's the steady-state goal — every additional drift finding from here is genuine signal of code-vs-catalog divergence.
+
+### Patch Changes
+
+- Updated dependencies [9fbbc3a]
+  - autotel-subscribers@33.0.0
+
 ## 1.0.0
 
 ### Minor Changes

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Jag Reehal 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -13,12 +13,13 @@ the code actually does at runtime.
 > on what channel. This package reads that snapshot and compares it to
 > your catalog.
 
-Two tools:
+Three tools:
 
-| Command     | Mode      | What it does                                                                                                                                                                              |
-| ----------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **`drift`** | read-only | Diffs the catalog against a snapshot. Reports findings as Markdown, JSON, or plain text. The PR check that catches "you added an event but forgot to document it."                        |
-| **`stamp`** | write     | Writes a runtime evidence block (counts, last-seen, field paths) into each event's `index.mdx` between idempotent markers. Keeps the static catalog page reflecting production behaviour. |
+| Command        | Mode      | What it does                                                                                                                                                                              |
+| -------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **`drift`**    | read-only | Diffs the catalog against a snapshot. Reports findings as Markdown, JSON, or plain text. The PR check that catches "you added an event but forgot to document it."                        |
+| **`generate`** | write     | Scaffolds EventCatalog resources from a snapshot: services, events, channels, inferred JSON Schemas, and producer↔event↔channel relationships.                                            |
+| **`stamp`**    | write     | Writes a runtime evidence block (counts, last-seen, field paths) into each event's `index.mdx` between idempotent markers. Keeps the static catalog page reflecting production behaviour. |
 
 Both share inputs: an autotel snapshot JSON file and an EventCatalog
 directory. Both ship a versioned JSON summary you can gate CI on. The
@@ -36,9 +37,11 @@ To keep the scope tight:
   This package only consumes them.
 - **Does not run any web server or dashboard.** Live dashboards live in
   example apps. This package is a CLI plus library plus action.
-- **Does not infer contract schemas from payload samples.** Field-path drift
-  is set-difference on dotted paths. Type/value drift is checked only against
-  declared schema constraints, not inferred contracts.
+- **Does not infer drift contracts from payload samples during `drift`.**
+  Field-path drift is set-difference on dotted paths. Type/value drift is
+  checked only against declared schema constraints. (`generate` can scaffold
+  schemas from snapshot evidence; `drift` still compares against declared
+  schemas.)
 - **Does not modify catalog files outside the stamp markers.** Everything
   the `stamp` command writes is between `<!-- autotel:stamp-start -->`
   and `<!-- autotel:stamp-end -->`. Outside those markers is yours.
@@ -66,11 +69,75 @@ autotel-eventcatalog drift \
   --fail-on-drift     # exit 1 on drift; wire this into CI
 ```
 
+```bash
+autotel-eventcatalog generate \
+  --snapshot ./services/test/snapshot.json \
+  --catalog ./catalog
+```
+
+Re-runnable: existing catalog files are skipped, never overwritten. New
+events / services / channels from the snapshot are added on top.
+Producer (`sends`), consumer (`receives`), and channel routing edges
+are wired automatically. Pass `--dry-run` to preview, `--edges-only`
+to re-sync relationships without touching resource bodies, or
+`--version 2.0.0` to override the default version for newly created
+resources.
+
+### Schema sources
+
+When `generate` writes an event's `schema.json`, it picks the schema
+in this order:
+
+1. **Declared schema** — if the snapshot's event observation carries a
+   `schema.jsonSchema` (recorded by `ArchitectureSnapshotSubscriber`
+   when a `track()` call was made through [`defineEvent`](#defineevent-zod-schemas-at-the-call-site)),
+   that schema is used verbatim.
+2. **Inferred schema** — otherwise, the schema is inferred from observed
+   `fieldStats` (runtime types per dotted path) as a fallback. This
+   gets a new project from zero to a usable catalog on the first run;
+   `defineEvent` is the path to a robust, single-source-of-truth
+   contract once you're ready.
+
+The choice is recorded in the operations log and the `generate-summary`
+JSON envelope as `schemaSource: 'declared' | 'inferred'`, so CI can
+surface adoption.
+
+#### `defineEvent`: Zod schemas at the call site
+
+In your service code, replace bare `track('event.name', payload)` calls
+with `defineEvent` and a Zod schema:
+
+```typescript
+import { defineEvent } from 'autotel';
+import { z } from 'zod';
+
+export const orderPlacedEvent = defineEvent(
+  'order.placed',
+  z.object({
+    orderId: z.string(),
+    customerId: z.string(),
+    totalCents: z.number(),
+    items: z.array(z.object({ sku: z.string(), quantity: z.number() })),
+  }),
+  { toJsonSchema: (schema) => z.toJSONSchema(schema) },
+);
+
+// At the call site:
+orderPlacedEvent.track({ orderId, customerId, totalCents, items });
+```
+
+The schema is now the single source of truth: TypeScript catches drift
+at compile time, `safeParse` validates payloads at runtime, and
+`ArchitectureSnapshotSubscriber` carries the JSON Schema forward into
+the snapshot — so `generate` writes the _same_ schema your code
+enforces, no inference guesswork.
+
 ### From code
 
 ```typescript
 import {
   loadSnapshot,
+  generateCatalogFromSnapshot,
   readCatalogState,
   diffCatalogAgainstSnapshot,
   renderMarkdown,
@@ -83,6 +150,12 @@ import {
 } from 'autotel-eventcatalog';
 
 const snapshot = await loadSnapshot('./services/test/snapshot.json');
+await generateCatalogFromSnapshot({
+  snapshot,
+  catalogPath: './catalog',
+  dryRun: false,
+});
+
 const catalog = await readCatalogState('./catalog');
 const report = diffCatalogAgainstSnapshot(snapshot, catalog);
 
@@ -93,34 +166,21 @@ if (hasDrift(report)) process.exit(1);
 
 ### What a run actually prints
 
-The example app in this monorepo (`apps/example-eventcatalog`) ships with
-two deliberate drift conditions so the CLI has something real to find.
-Running `pnpm catalog:drift` there prints:
+After the example app's services have been migrated to `defineEvent`
+and the catalog's schemas align with what the code emits, running
+`pnpm catalog:drift` from `apps/example-eventcatalog` prints:
 
 ```
 # Architecture drift report
 
-## Events documented but never observed
-
-- `PaymentFailed`
-
-## Field-path drift
-
-### `recommendation.generated`
-
-**Extra fields in payloads (not in declared schema):**
-
-- `personalization_seed`
-
-Drift detected in current snapshot.
+No drift detected. Catalog and runtime agree.
 ```
 
-Exit code 1, CI fails. Two genuine findings, zero noise:
-
-- **`PaymentFailed` documented but never observed** — the failure path
-  isn't exercised by the integration tests. A real coverage gap.
-- **`personalization_seed` extra field** — the service emits it; the
-  catalog schema doesn't declare it. Real schema drift.
+Exit code 0, CI passes. That's the steady-state goal: the catalog and
+the running code make the same claims and the drift detector confirms
+it. Every additional finding is genuine signal of code-vs-catalog
+divergence — a new event added without a schema, a removed producer
+still listed, a schema field that no longer ships.
 
 Type drift handles the JSON Schema ↔ JavaScript impedance mismatch
 deliberately: a declared `integer` accepts an observed `number` at the
@@ -128,6 +188,12 @@ type level, then sample values are checked against `Number.isInteger`
 so a runtime `1.5` against a declared `integer` still flags. No false
 positives, no swept-under-the-rug genuine signal.
 
+The example app exercises both the happy path and the payment-failure
+path so the snapshot covers every documented event. Value drift would
+fire if, for instance, a `declineCode` outside the declared enum
+appeared at runtime — try replacing `card_declined` with something else
+in `build-snapshot.ts` and re-running to see it.
+
 ### As a GitHub Action
 
 The package ships a composite action so any repository can wire drift checking
@@ -211,20 +277,22 @@ What lands on the PR:
    a `valueDrift` entry. Both feed `countDriftReport` and the markdown
    renderer.
 
-A planned next step is first-class Zod metadata at `track()` call sites
-so contracts can be declared in code rather than only in catalog schemas.
+You can now pass first-class Zod metadata from `track()` call sites (via
+`defineEvent(...)` in `autotel`) so contracts can be declared in code and
+propagated into runtime snapshots.
 
 ## Public JSON contract
 
-Three versioned JSON shapes are shipped with the package as JSON Schema
+Four versioned JSON shapes are shipped with the package as JSON Schema
 files (`schemas/` directory). Downstream tooling (your own GitHub Actions,
 dashboards, Slack bots) should validate against these:
 
-| Schema                              | Emitted by                   | Consumed by                              |
-| ----------------------------------- | ---------------------------- | ---------------------------------------- |
-| `schemas/drift-report-v0.1.0.json`  | `drift --format json`        | Any downstream parser                    |
-| `schemas/drift-summary-v0.1.0.json` | `drift --summary-output ...` | CI gating, dashboards                    |
-| `schemas/stamp-summary-v0.1.0.json` | `stamp --summary-output ...` | "Did this PR forget to re-stamp?" checks |
+| Schema                                 | Emitted by                      | Consumed by                              |
+| -------------------------------------- | ------------------------------- | ---------------------------------------- |
+| `schemas/drift-report-v0.2.0.json`     | `drift --format json`           | Any downstream parser                    |
+| `schemas/drift-summary-v0.2.0.json`    | `drift --summary-output ...`    | CI gating, dashboards                    |
+| `schemas/stamp-summary-v0.1.0.json`    | `stamp --summary-output ...`    | "Did this PR forget to re-stamp?" checks |
+| `schemas/generate-summary-v0.1.0.json` | `generate --summary-output ...` | scaffold/edge generation auditing        |
 
 Every envelope carries a `spec: 'autotel-eventcatalog-…/vX.Y.Z'` field
 that downstream code can use to refuse unknown major versions. The shapes
@@ -234,7 +302,8 @@ changes them without bumping the spec version will fail CI.
 ## Renderers (advanced)
 
 `drift --format <name>` dispatches through a small renderer registry. The
-built-ins are `markdown`, `terminal`, `json`. The registry is exported
+built-ins are `markdown`, `terminal`, `json`, `eventcatalog-snapshot-diff`.
+The registry is exported
 from the library so applications and other tooling can add their own:
 
 ```typescript