autotel-eventcatalog 1.0.0

package/README.md

@@ -0,0 +1,307 @@
+# autotel-eventcatalog
+
+Keep your [EventCatalog](https://www.eventcatalog.dev) honest about what
+the code actually does at runtime.
+
+> **New here?** [autotel](https://github.com/jagreehal/autotel) is an
+> ergonomic OpenTelemetry wrapper for Node.js: you call `trace()`, `span()`
+> and `track(eventName, payload)` in your service code and it emits spans
+> and domain events. The `ArchitectureSnapshotSubscriber` from
+> `autotel-subscribers` listens to those events during a test run and
+> writes a `snapshot.json` describing every event that fired, the fields
+> in its payload, the runtime types of those fields, who produced it and
+> on what channel. This package reads that snapshot and compares it to
+> your catalog.
+
+Two tools:
+
+| Command     | Mode      | What it does                                                                                                                                                                              |
+| ----------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **`drift`** | read-only | Diffs the catalog against a snapshot. Reports findings as Markdown, JSON, or plain text. The PR check that catches "you added an event but forgot to document it."                        |
+| **`stamp`** | write     | Writes a runtime evidence block (counts, last-seen, field paths) into each event's `index.mdx` between idempotent markers. Keeps the static catalog page reflecting production behaviour. |
+
+Both share inputs: an autotel snapshot JSON file and an EventCatalog
+directory. Both ship a versioned JSON summary you can gate CI on. The
+`drift` command also ships as a one-line GitHub Action with a sticky PR
+comment.
+
+Same model as Pact, for event architectures.
+
+## What this package does NOT do
+
+To keep the scope tight:
+
+- **Does not produce snapshots.** Snapshots come from
+  [`autotel-subscribers`'s `ArchitectureSnapshotSubscriber`](../autotel-subscribers).
+  This package only consumes them.
+- **Does not run any web server or dashboard.** Live dashboards live in
+  example apps. This package is a CLI plus library plus action.
+- **Does not infer contract schemas from payload samples.** Field-path drift
+  is set-difference on dotted paths. Type/value drift is checked only against
+  declared schema constraints, not inferred contracts.
+- **Does not modify catalog files outside the stamp markers.** Everything
+  the `stamp` command writes is between `<!-- autotel:stamp-start -->`
+  and `<!-- autotel:stamp-end -->`. Outside those markers is yours.
+
+## Install
+
+```bash
+pnpm add -D autotel-eventcatalog
+```
+
+Requires that your services use autotel and `ArchitectureSnapshotSubscriber`
+from `autotel-subscribers/architecture-snapshot` to produce a snapshot.
+
+## Use
+
+### From the CLI
+
+```bash
+autotel-eventcatalog drift \
+  --snapshot ./services/test/snapshot.json \
+  --catalog ./catalog \
+  --output ./drift.md \
+  --summary-output ./drift-summary.json \
+  --policy all \
+  --fail-on-drift     # exit 1 on drift; wire this into CI
+```
+
+### From code
+
+```typescript
+import {
+  loadSnapshot,
+  readCatalogState,
+  diffCatalogAgainstSnapshot,
+  renderMarkdown,
+  renderTerminal, // plain-text, for Slack / log files / non-MD terminals
+  renderJson, // versioned envelope for downstream tooling
+  countDriftReport, // per-category counts that match the dashboard hero badge
+  hasDrift,
+  stampCatalog, // write runtime evidence into your catalog's event mdx
+  buildStampSummary, // count of inserts / replaces / no-ops / skipped
+} from 'autotel-eventcatalog';
+
+const snapshot = await loadSnapshot('./services/test/snapshot.json');
+const catalog = await readCatalogState('./catalog');
+const report = diffCatalogAgainstSnapshot(snapshot, catalog);
+
+console.log(renderMarkdown(report));
+console.log('findings:', countDriftReport(report).total);
+if (hasDrift(report)) process.exit(1);
+```
+
+### What a run actually prints
+
+The example app in this monorepo (`apps/example-eventcatalog`) ships with
+two deliberate drift conditions so the CLI has something real to find.
+Running `pnpm catalog:drift` there prints:
+
+```
+# Architecture drift report
+
+## Events documented but never observed
+
+- `PaymentFailed`
+
+## Field-path drift
+
+### `recommendation.generated`
+
+**Extra fields in payloads (not in declared schema):**
+
+- `personalization_seed`
+
+Drift detected in current snapshot.
+```
+
+Exit code 1, CI fails. Two genuine findings, zero noise:
+
+- **`PaymentFailed` documented but never observed** — the failure path
+  isn't exercised by the integration tests. A real coverage gap.
+- **`personalization_seed` extra field** — the service emits it; the
+  catalog schema doesn't declare it. Real schema drift.
+
+Type drift handles the JSON Schema ↔ JavaScript impedance mismatch
+deliberately: a declared `integer` accepts an observed `number` at the
+type level, then sample values are checked against `Number.isInteger`
+so a runtime `1.5` against a declared `integer` still flags. No false
+positives, no swept-under-the-rug genuine signal.
+
+### As a GitHub Action
+
+The package ships a composite action so any repository can wire drift checking
+into its PR pipeline with a single step:
+
+```yaml
+# .github/workflows/eventcatalog-drift.yml
+name: eventcatalog drift
+on:
+  pull_request:
+    branches: [main]
+jobs:
+  drift:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+    steps:
+      - uses: actions/checkout@v4
+        with: { fetch-depth: 0 } # so the action can read the base branch
+
+      # Produce the snapshot however you like; typically by running your
+      # integration tests with ArchitectureSnapshotSubscriber wired in.
+      - run: pnpm install --frozen-lockfile
+      - run: pnpm services:snapshot
+
+      - uses: jagreehal/autotel-eventcatalog@v0
+        with:
+          snapshot: ./services/test/snapshot.json
+          catalog: ./catalog
+          base-ref: origin/${{ github.base_ref }} # compare to PR base
+          fail-on-drift: true # fail the PR on new drift
+          comment-on-pr: true # post a sticky comment
+```
+
+CLI policy modes:
+
+- `--policy all`: fail on any drift in the current snapshot.
+- `--policy new-only`: fail only on drift introduced vs `--base-snapshot`.
+
+What lands on the PR:
+
+- A sticky comment titled "Architecture drift: what this change introduces"
+  with sections for new events, removed events, field-path drift, type drift,
+  value drift, and drift the PR resolved.
+- The check fails only when this PR introduces _new_ drift. Pre-existing
+  drift is reported for context but does not block.
+- The drift report is also written to `$RUNNER_TEMP/autotel-eventcatalog-drift.md`
+  and printed in the job log.
+
+## What gets caught
+
+| Drift class                              | Example finding                                            |
+| ---------------------------------------- | ---------------------------------------------------------- |
+| **Events observed but undocumented**     | `order.cancelled` emitted by code; no entry in catalog     |
+| **Events documented but never observed** | `LegacyEvent` in catalog; never seen in tests              |
+| **Field-path drift (extra)**             | `personalization_seed` in payload; not declared in schema  |
+| **Field-path drift (missing)**           | `customerId` declared in schema; never present in payloads |
+| **Type drift**                           | `amount` declared `number`; observed `string`              |
+| **Value drift (enum mismatch)**          | `status: "placed"` observed; schema enum excludes it       |
+| **Services observed but undocumented**   | `OrdersService` is a producer; no service page             |
+| **Channels observed but undocumented**   | `orders.events` carries messages; no channel page          |
+
+**How type and value drift work in practice.**
+
+1. `ArchitectureSnapshotSubscriber` records `fieldStats` per event during
+   a test run — for each dotted path it captures the runtime types it
+   saw (`string`, `number`, `object`, …) and up to 20 primitive sample
+   values. Verified by
+   [`snapshot-fieldstats.integration.test.ts`](../../apps/example-eventcatalog/services/test/snapshot-fieldstats.integration.test.ts),
+   which runs the real `placeOrder` → `handleOrderPlaced` →
+   `handlePaymentCaptured` → `generateRecommendation` flow and asserts
+   `fieldStats.totalCents.types` contains `"number"`,
+   `fieldStats.currency.sampleValues` contains `"GBP"`, etc.
+2. `readCatalogState` parses each event's JSON Schema (via `schemaPath`
+   in the catalog frontmatter) into a map of declared `types` and `enum`
+   values per path.
+3. `diffCatalogAgainstSnapshot` cross-joins the two: any path whose
+   observed runtime type is not in the declared types becomes a
+   `typeDrift` entry; any sample value not in the declared enum becomes
+   a `valueDrift` entry. Both feed `countDriftReport` and the markdown
+   renderer.
+
+A planned next step is first-class Zod metadata at `track()` call sites
+so contracts can be declared in code rather than only in catalog schemas.
+
+## Public JSON contract
+
+Three versioned JSON shapes are shipped with the package as JSON Schema
+files (`schemas/` directory). Downstream tooling (your own GitHub Actions,
+dashboards, Slack bots) should validate against these:
+
+| Schema                              | Emitted by                   | Consumed by                              |
+| ----------------------------------- | ---------------------------- | ---------------------------------------- |
+| `schemas/drift-report-v0.1.0.json`  | `drift --format json`        | Any downstream parser                    |
+| `schemas/drift-summary-v0.1.0.json` | `drift --summary-output ...` | CI gating, dashboards                    |
+| `schemas/stamp-summary-v0.1.0.json` | `stamp --summary-output ...` | "Did this PR forget to re-stamp?" checks |
+
+Every envelope carries a `spec: 'autotel-eventcatalog-…/vX.Y.Z'` field
+that downstream code can use to refuse unknown major versions. The shapes
+are locked by golden contract tests inside this package: a PR that
+changes them without bumping the spec version will fail CI.
+
+## Renderers (advanced)
+
+`drift --format <name>` dispatches through a small renderer registry. The
+built-ins are `markdown`, `terminal`, `json`. The registry is exported
+from the library so applications and other tooling can add their own:
+
+```typescript
+import { RENDERERS, type Renderer } from 'autotel-eventcatalog';
+
+const sarifRenderer: Renderer = {
+  name: 'sarif',
+  description: 'Static Analysis Results Interchange Format',
+  renderReport(report) {
+    /* ... */ return '';
+  },
+  renderDelta(delta) {
+    /* ... */ return '';
+  },
+};
+// Programmatic consumers can plug it in. The CLI is also extensible via
+// a follow-up release that supports loading renderers from a config file.
+```
+
+The core diff and policy modules are renderer-agnostic; adding a new
+output target should never require touching `diff.ts` or `policy.ts`.
+
+## What writes to the catalog
+
+`drift` is **read-only**. It diffs the snapshot against the catalog and
+reports findings. It never modifies catalog files.
+
+`stamp` is the **write path**. By design, it injects a runtime evidence
+block into each event mdx between idempotent markers
+(`<!-- autotel:stamp-start -->` / `<!-- autotel:stamp-end -->`). Re-runs
+replace the content between the markers; nothing outside the markers is
+touched. Run with `--dry-run` to preview the plan first.
+
+Pass `--summary-output stamp.json` to write a versioned summary file
+(`autotel-eventcatalog-stamp-summary/v0.1.0`) describing how many files
+were inserted, replaced, no-op'd, or skipped. Useful for CI checks that
+need to gate on "did this PR forget to re-stamp?".
+
+## Working example
+
+See [`apps/example-eventcatalog`](../../apps/example-eventcatalog) in the
+monorepo. It has a working e-commerce catalog and a snapshot. Running
+`pnpm catalog:drift` from that app prints a drift report that surfaces
+two findings, including a deliberately-introduced `personalization_seed`
+field that the schema does not declare.
+
+## Documentation
+
+| If you want to…                                        | Read                                                 |
+| ------------------------------------------------------ | ---------------------------------------------------- |
+| Consume the JSON output from your own tool             | [`docs/CONTRACT.md`](docs/CONTRACT.md)               |
+| Write a custom renderer (SARIF, Slack, your dashboard) | [`docs/EXTENDING.md`](docs/EXTENDING.md)             |
+| Debug a CLI exit code or a CI failure                  | [`docs/TROUBLESHOOTING.md`](docs/TROUBLESHOOTING.md) |
+| Upgrade between versions safely                        | [`docs/UPGRADING.md`](docs/UPGRADING.md)             |
+| See what shipped in each release                       | [`CHANGELOG.md`](CHANGELOG.md)                       |
+| Contribute to the package                              | [`CONTRIBUTING.md`](CONTRIBUTING.md)                 |
+| Validate the published JSON shapes                     | [`schemas/README.md`](schemas/README.md)             |
+
+## Used by
+
+_If you adopt this in production, open a PR adding your team to this
+section. We'd like to know who depends on the contract before shipping
+breaking changes, and seeing a name here helps future adopters gauge
+the project's maturity._
+
+- _no public adopters yet; be the first_
+
+## License
+
+MIT.

package/action.yml

@@ -0,0 +1,155 @@
+name: 'autotel-eventcatalog drift'
+description: |
+  Diff your autotel architecture snapshot against an EventCatalog and surface
+  drift on the pull request. With base-ref input, only drift the PR introduces
+  fails the check — pre-existing drift is reported but ignored.
+
+branding:
+  icon: 'activity'
+  color: 'orange'
+
+inputs:
+  snapshot:
+    description: 'Path to the architecture snapshot JSON (the PR head version).'
+    required: true
+  catalog:
+    description: 'Path to the EventCatalog directory (containing eventcatalog.config.* and domains/).'
+    required: true
+  base-ref:
+    description: |
+      Git ref to read the baseline snapshot from (typically the PR base branch,
+      e.g. `origin/main`). When set, the action reports only drift the PR
+      introduces — pre-existing drift is shown for context but does not fail.
+    required: false
+    default: ''
+  fail-on-drift:
+    description: 'Fail the check when drift is detected. With base-ref, fails only on *new* drift.'
+    required: false
+    default: 'true'
+  comment-on-pr:
+    description: 'Post the drift report as a sticky comment on the pull request.'
+    required: false
+    default: 'true'
+  comment-header:
+    description: 'Header tag for the sticky comment (use this if you have multiple drift checks).'
+    required: false
+    default: 'autotel-eventcatalog'
+  package-version:
+    description: |
+      Semver range for autotel-eventcatalog. Defaults to `^0` so a new
+      major release won't silently land on your CI — pin to an exact
+      version (e.g. `0.1.0`) for fully reproducible runs.
+    required: false
+    default: '^0'
+  continue-on-comment-failure:
+    description: |
+      When `true`, a failure of the sticky-comment step (e.g. GitHub API
+      rate limit, transient 5xx, permissions) is downgraded to a warning
+      and the workflow keeps running. The drift check itself is
+      unaffected. Default `true` because a missed comment is annoying;
+      a failed job because of a missed comment is worse.
+    required: false
+    default: 'true'
+
+outputs:
+  drift-detected:
+    description: '"true" if any drift was detected, "false" otherwise.'
+    value: ${{ steps.run.outputs.drift-detected }}
+  report-path:
+    description: 'Path to the generated markdown drift report.'
+    value: ${{ steps.run.outputs.report-path }}
+
+runs:
+  using: composite
+  steps:
+    - name: Resolve base snapshot
+      id: base
+      if: ${{ inputs.base-ref != '' }}
+      shell: bash
+      run: |
+        set -euo pipefail
+        BASE_SNAPSHOT="$RUNNER_TEMP/autotel-eventcatalog-base-snapshot.json"
+        if git show "${{ inputs.base-ref }}:${{ inputs.snapshot }}" > "$BASE_SNAPSHOT" 2>/dev/null; then
+          echo "Found base snapshot at ${{ inputs.base-ref }}:${{ inputs.snapshot }}"
+          echo "path=$BASE_SNAPSHOT" >> "$GITHUB_OUTPUT"
+        else
+          echo "::notice::No base snapshot at ${{ inputs.base-ref }}:${{ inputs.snapshot }} — first run on this branch?"
+          echo "path=" >> "$GITHUB_OUTPUT"
+        fi
+
+    - name: Run drift check
+      id: run
+      shell: bash
+      env:
+        SNAPSHOT: ${{ inputs.snapshot }}
+        CATALOG: ${{ inputs.catalog }}
+        BASE_SNAPSHOT_PATH: ${{ steps.base.outputs.path }}
+        PACKAGE_VERSION: ${{ inputs.package-version }}
+      run: |
+        set -euo pipefail
+        REPORT="$RUNNER_TEMP/autotel-eventcatalog-drift.md"
+        SUMMARY="$RUNNER_TEMP/autotel-eventcatalog-summary.json"
+        echo "report-path=$REPORT" >> "$GITHUB_OUTPUT"
+
+        # Always pass --fail-on-drift internally so the exit code is the
+        # authoritative drift signal. The user-facing `fail-on-drift` input
+        # only controls whether we re-emit that failure as a workflow
+        # failure further down — the action itself needs the signal either
+        # way (for the drift-detected output, the sticky comment, etc.).
+        POLICY="all"
+        ARGS=(--snapshot "$SNAPSHOT" --catalog "$CATALOG" --output "$REPORT" --summary-output "$SUMMARY" --fail-on-drift)
+        if [ -n "${BASE_SNAPSHOT_PATH:-}" ]; then
+          ARGS+=(--base-snapshot "$BASE_SNAPSHOT_PATH")
+          POLICY="new-only"
+        fi
+        ARGS+=(--policy "$POLICY")
+
+        # Run the CLI — non-zero exit means drift was detected.
+        set +e
+        npx --yes "autotel-eventcatalog@$PACKAGE_VERSION" drift "${ARGS[@]}"
+        STATUS=$?
+        set -e
+
+        if [ $STATUS -ne 0 ] && [ $STATUS -ne 1 ]; then
+          # Anything else (2 = bad args, etc.) is a hard error; surface it.
+          echo "::error::autotel-eventcatalog drift CLI failed with exit $STATUS"
+          exit $STATUS
+        fi
+
+        if [ ! -f "$SUMMARY" ]; then
+          echo "::error::autotel-eventcatalog did not produce a summary output."
+          exit 1
+        fi
+        DRIFT_DETECTED="$(node -e "const fs=require('fs');const s=JSON.parse(fs.readFileSync(process.argv[1],'utf8'));process.stdout.write(s.shouldFail ? 'true':'false')" "$SUMMARY")"
+        echo "drift-detected=$DRIFT_DETECTED" >> "$GITHUB_OUTPUT"
+
+        # Always show the report in the job log for easy review.
+        echo "::group::Drift report"
+        cat "$REPORT"
+        echo "::endgroup::"
+
+    # Sticky-comment action pinned to a specific commit SHA. This is a
+    # supply-chain control — a future malicious tag wouldn't affect anyone
+    # using this action. Update the SHA deliberately when bumping the
+    # external action's version. Currently pinned to v2.9.4 (2024-12-03).
+    - name: Post sticky PR comment
+      id: comment
+      if: ${{ inputs.comment-on-pr == 'true' && github.event_name == 'pull_request' }}
+      continue-on-error: ${{ inputs.continue-on-comment-failure == 'true' }}
+      uses: marocchino/sticky-pull-request-comment@52423e01640425a022ef5fd42c6fb5f633a02728
+      with:
+        header: ${{ inputs.comment-header }}
+        path: ${{ steps.run.outputs.report-path }}
+
+    - name: Warn on comment failure
+      if: ${{ steps.comment.outcome == 'failure' }}
+      shell: bash
+      run: |
+        echo "::warning::Could not post the drift report as a PR comment. The drift check itself completed and the report is available at ${{ steps.run.outputs.report-path }}."
+
+    - name: Fail on drift
+      if: ${{ inputs.fail-on-drift == 'true' && steps.run.outputs.drift-detected == 'true' }}
+      shell: bash
+      run: |
+        echo "::error::autotel-eventcatalog detected drift between the catalog and the snapshot."
+        exit 1