@walkeros/core 4.1.0-next-1778668930820 → 4.1.0

package/CHANGELOG.md

@@ -0,0 +1,1089 @@
+# @walkeros/core
+
+## 4.1.0
+
+### Minor Changes
+
+- e155ff8: Cache reads through `checkCache` are now correct against async stores
+  (filesystem, Redis, any store with an async `get`). Previously a custom async
+  store could silently miss the cache.
+
+  `checkCache` returns a Promise. External callers must add `await`.
+
+- e155ff8: Collector and destination buffers are now size-bounded with FIFO
+  drop-oldest eviction. Defaults: collector `queueMax: 1000`, destination
+  `queueMax: 1000`, destination `dlqMax: 100`. Set either knob to override per
+  scope. Drop counts surface in `collector.status.dropped` and
+  `collector.status.destinations[id]`.
+- b276173: **Breaking:** `code: "<exportName>"` is no longer accepted on any
+  step. Replace with `import: "<exportName>"` alongside `package`.
+
+  **New:** Every step (source, transformer, destination, store) accepts
+  `import?: string`. With `package`, it selects a named export. `package` alone
+  still loads the default export. Inline code stays
+  `code: { push, type?, init? }`. Empty steps are valid no-ops. `flow_validate`
+  and the CLI bundler raise `OBSOLETE_CODE_STRING` on the legacy shape with a
+  precise rename hint.
+
+- dd9f5ad: Pass-through transformer steps + closed-schema validation.
+
+  **Validation:** `validateTransformerEntry` in `@walkeros/core` is now the
+  single source of truth. Bundler, `flow_validate`, and collector runtime all
+  delegate. Closed schema: unknown top-level keys are errors. `code` + `package`
+  together is a `CONFLICT`.
+
+  **Pass-through steps:** A transformer entry with no `code` and no `package` is
+  valid; the collector synthesizes its push. Three variants:
+  - before/next chain only (named hop)
+  - cache only (e.g. dedup)
+  - mapping only (event-to-event transform via `Mapping.Config`)
+
+  **Mapping at the transformer position:** new `mapping?: Mapping.Config` field
+  on `Transformer.Config` / `InitTransformer`. Same shape as
+  `Destination.Config.mapping`, event-to-event semantic. `data` / `silent` are
+  ignored at the transformer position with a one-time warning.
+
+  **Engine tag:** synthesized instance now uses `type: 'pass'` (was `'path'`).
+  Hard cut.
+
+  **Runtime fixes:**
+  - `compileNext` handles mixed-shape `next` arrays (`["a", { case }]`) via a
+    new `'sequence'` variant.
+  - A destination's `before` referencing a pass-through transformer now walks
+    that transformer's own `before` / `next`.
+  - `cache.stop: true` at a pre-collector transformer halts the pipeline
+    (matches `cache.mdx`).
+
+  **Migration:** Typo keys on a step now fail validation.
+  `instance.type === 'path'` consumers must read `'pass'`. `runTransformerChain`
+  consumers should branch on the new `stopped` flag.
+
+- adeebea: Route grammar: rename `case` to `one` (first-match dispatch) and add
+  `many` (all-match parallel fan-out, pre-collector only). `many` terminates the
+  main chain and is rejected at post-collector positions (`destination.before`,
+  `destination.next`); use multiple destinations for post-collector fan-out.
+  `RouteCaseConfig` is renamed to `RouteOneConfig`; no aliases.
+- 13aaeaa: `Source.Context` no longer exposes `setIngest` or `setRespond`.
+  Server sources handling concurrent inbound requests must call
+  `context.withScope(rawScope, respond, body)` to bind per-request ingest and
+  respond. Browser and other single-scope sources keep working without changes.
+- e800974: `Status.dropped` is now keyed by stepId, so operators can see at a
+  glance which step dropped events. Read with
+  `status.dropped["collector"]?.queue` or
+  `status.dropped["destination.<id>"]?.queue` / `.dlq`, or build the key with
+  the new `stepId()` helper exported from `@walkeros/core`. Breaking change: the
+  previous flat shape (`status.dropped.queue` / `.queuePush` / `.dlq`) and the
+  per-destination `dropped` field on `DestinationStatus` are removed.
+- adeebea: Add `Flow.Store.cache` for store-level caching: read-through +
+  write-through wrapper with single-flight dedup, recursive composition via
+  `cache.store`, and per-wrapper counters. `CacheRule` is now a discriminated
+  union (`EventCacheRule | StoreCacheRule`); schema rejects inert fields in
+  store contexts.
+
+  Built-in `__cache` upgraded with LRU, `maxEntries: 10000`, batched eviction,
+  and active TTL sweep.
+
+  **Breaking:** `@walkeros/store-memory` is removed. Its logic is absorbed into
+  `__cache`. Migration: drop the store declaration, or omit `cache.store` to use
+  the built-in tier. `flow_validate` flags legacy references.
+
+- 058f7ed: Add `validateJsonSchema` / `validateEventsJsonSchema` exports for
+  step-level validation config, promote the validate and no-`many` route schemas
+  to direct exports, and add optional `nodeType` / `subPath` cursor fields to
+  `IntelliSenseContext` for context-scoped autocomplete.
+- 28a8ac2: Add step-level `validate?` primitive on every walkerOS step.
+  `validate:` is a declarative description of validation intent, like `cache` or
+  `consent`. Consumers decide how to enforce.
+
+  Restructure `Flow.ContractRule` to a uniform
+  `{ extends?, tagging?, description?, events?, schema? }` shape. A single
+  agnostic JSON Schema replaces the typed section fields (`globals`, `context`,
+  `custom`, `user`, `consent`); standard event field names live inside
+  `schema.properties.<name>`. `extends` resolves `schema` via additive
+  deep-merge.
+
+  Contracts are a description and governance concept: tooling, MCP, and humans
+  read them. Runtime enforcement is the consumer's call.
+
+- fd6076e: Walker commands `destination`, `hook`, and `on` now take a single
+  Init object: `elb('walker destination', { code, config })`,
+  `elb('walker hook', { name, fn })`, `elb('walker on', { type, rules })`. The
+  previous positional forms and the `{ push }` shorthand are removed; the
+  `options` argument is gone from `collector.command`, `addDestination`, and
+  `commonHandleCommand`.
+
+### Patch Changes
+
+- e800974: Fix `getByPath` silently returning undefined for objects created in a
+  different realm (Node `http.IncomingMessage`, `vm` contexts, worker threads,
+  iframes). The internal `instanceof Object` guard is replaced with a
+  cross-realm-safe check, so dot-notation paths now extract fields from native
+  request objects and other cross-realm sources.
+- 1a8f2d7: Fix `resolveContracts` so a child contract that uses `extends`
+  inherits the parent's `tagging` when it does not redeclare it. Previously the
+  parent's `tagging` was silently dropped, which corrupted contract version
+  tracking for anyone building on a base contract.
+- 1a8f2d7: Flow v4 routing & cache cleanup.
+
+  **Cache:**
+  - `cache.full` is renamed to `cache.stop`. Search-and-replace.
+  - `cacheRule.match` is now optional. Omitted means always-match. The literal
+    `'*'` is dropped from the schema and the TypeScript types; `compileMatcher`
+    still tolerates the string at runtime for migration.
+  - New `cache.namespace?: string` field. Omit to write keys directly to the
+    store. Same store + same key + same namespace = same cache entry.
+  - Implicit per-step namespace prefixes (`s:`, `t:`, `d:`) are removed. If you
+    relied on them to separate same-keyed caches across
+    sources/transformers/destinations using the same store, set
+    `cache.namespace` explicitly.
+
+  **Routing:**
+  - Unified recursive `Route` type. A Route is `string | Route[] | RouteConfig`.
+  - New `case` operator replaces the legacy `Route[]` first-match shape. The
+    legacy shape is compiled as an implicit `{ case: [...] }` for runtime
+    compatibility, but new configs should use `case` explicitly.
+  - `RouteConfig` is a disjoint union enforced at the TypeScript type level via
+    `never` fields: a single RouteConfig sets at most one of `next` / `case`. A
+    bare `{ match }` is a gate (pass-through when the match fires, fall-through
+    when it fails). JSON Schema validation currently emits `anyOf` and does not
+    enforce disjointness at runtime — see follow-up notes.
+  - Sequence sugar (`next: [A, B, C]`) is preserved.
+
+  **Path:**
+  - A transformer entry with no `code` is a `path` — a code-less passthrough.
+    The engine synthesizes `(e) => ({ event: e })`. Use paths to name and share
+    `before` chains across destinations. Validation: a path must declare at
+    least one of `package`, `before`, `next`, or `cache`.
+
+  **Schema & tooling:**
+  - Updated Zod schemas (cache, route, matcher).
+  - Updated MCP tool descriptions and resource references.
+  - Updated `flow_validate` to enforce the new constraints (`EMPTY_TRANSFORMER`
+    error code added).
+
+  **Migration:** Hard cut at the schema/type level. Configs using `cache.full`
+  will fail validation — rename to `stop`. Configs using `match: "*"` will fail
+  validation — omit `match`. Configs using `Route[]` first-match still work at
+  runtime (compiled as implicit `case`) but new configs should use `case`
+  explicitly.
+
+  `$schema: "v4"` is preserved. No version bump.
+
+- c60ef35: Remove unused legacy fields `batchFn` and `batched` from
+  `Mapping.Rule`. Batch state lives on the destination via `BatchRegistry`,
+  never on mapping rules. No runtime impact.
+- e800974: Internal pipeline failures in mapping, source startup, transformer
+  init, and destination init now log via the scoped logger and increment
+  `collector.status.failed`. Previously silent. User-supplied callbacks (mapping
+  `condition`/`fn`/`validate`, `on` subscriptions) log on throw but do not
+  affect `status.failed`. A source whose `init()` throws now stays
+  `config.init === false` instead of being marked initialized.
+- e800974: Add typed accessors `Source.getSource`, `Destination.getDestination`,
+  `Transformer.getTransformer`, `Store.getStore`. Each takes a collector and a
+  step id and returns the registered instance with its declared generic
+  recovered, replacing the `Elb.Fn`-collapsed shape that the bag's index
+  signature exposes on read.
+
+  Callers (mainly tests and integrations that invoke a step's raw `push` through
+  the collector) no longer need `as any` / `as (rawData: X) => ...` casts at
+  this boundary. Each helper throws `<Kind> not found: <id>` for unknown ids. No
+  runtime behavior change.
+
+## 4.0.2
+
+### Patch Changes
+
+- a6a0ea7: Rename routing types: `NextRule` to `Route`, `Next` to `RouteSpec`
+  (Zod schemas and `MatcherNext*` IDs renamed in step). Widen
+  `Flow.*.before/next` to `RouteSpec` so `Route[]` conditional routing
+  type-checks at the JSON layer. Fix the CLI bundler dropping `Route[]` data via
+  a narrowing cast on the inline path. Hard cut, no aliases; flow.json shape
+  unchanged.
+
+## 4.0.1
+
+### Patch Changes
+
+- 381dfe7: Add an optional `setup` lifecycle to destinations, sources, and
+  stores.
+
+  Each package may now implement `setup?: SetupFn` to provision external
+  resources (BigQuery datasets and tables, Pub/Sub topics and subscriptions,
+  SQLite tables, webhook registrations, etc.). Setup is triggered only by the
+  new `walkeros setup <kind>.<name>` CLI command, never automatically by the
+  runtime, push, or deploy. Idempotency, ordering, and error semantics are the
+  package's responsibility; the framework provides the type slot, the CLI
+  invocation, and a `resolveSetup(value, defaults)` helper.
+
+  `LifecycleContext<C, E>` is the new shared context type used by both `setup`
+  and `destroy`. `DestroyContext` remains as a deprecated type alias for one
+  minor cycle. The `Types` bundle on `Destination`, `Source`, and `Store` gains
+  a 5th/6th/4th positional slot for setup options; existing aliases compile
+  unchanged because the slot defaults to `unknown`.
+  `Config<T>.setup?: boolean | SetupOptions<T>` is added across all three kinds
+  and validated by the corresponding Zod `ConfigSchema` plus the flow component
+  schemas in `@walkeros/core/schemas/flow.ts`.
+
+  CLI:
+  - `walkeros setup <kind>.<name>` runs a single component's `setup()` function.
+  - `<kind>` is `source`, `destination`, or `store` (transformers have no
+    provisioning).
+  - `--config <path>` (default `./flow.json`), `--flow <name>` for multi-flow
+    configs, plus standard `--json` / `--verbose` / `--silent`.
+  - Exit 0 on success or skip; non-zero on failure. Skip narration covers three
+    cases: no `setup()` on the package, `config.setup === false`, or
+    `config.setup` unset.
+  - When the package's `setup()` returns a non-undefined value, the CLI emits it
+    as JSON on stdout for `jq` piping.
+
+- 1524275: Source lifecycle redesign: factory + eager `init` + collector-gated
+  `on()`
+
+  Source factories must now be side-effect-free. The collector calls
+  `Instance.init()` on each source eagerly after all factories register.
+  `require` no longer gates code execution. It gates `on(type)` delivery (events
+  queue in `Instance.queueOn` until the source is started, then replay).
+  `collector.pending.sources` has been removed; per-source state lives on
+  `Source.Instance` (`queueOn`) and `Source.Config` (`init`, `require`).
+
+  Migration: any source factory with side effects (queue draining, walker
+  command emission, listener attachment) should move those into the returned
+  Instance's optional `init` method. Tests asserting on
+  `collector.pending.sources` should read `collector.sources[id]` and inspect
+  `config.init` / `config.require` instead.
+
+  Fixes the elbLayer queue replay clobbering fresh consent/user state,
+  late-activated sources missing `walker run`, and inter-source require chains
+  racing when a non-required source's init fired a state-mutating walker command
+  before later require-gated sources had been registered.
+
+- 03d7055: Merge `definitions` into `variables`. Single concept, single syntax
+  `$var.name(.deep.path)?`. Whole-string references preserve native type; inline
+  interpolation requires scalars. Deep paths and recursive resolution with cycle
+  detection now supported. `Flow.Definitions`, `Flow.Primitive`, and the `$def.`
+  reference syntax are removed.
+
+## 4.0.0
+
+### Major Changes
+
+- 93ea9c4: Event model v4: breaking changes to the `Event`, `Source`, and
+  `Entity` shapes.
+  - `event.id` is now a W3C span_id (16 lowercase hex chars), generated by the
+    collector. Reference: W3C Trace Context (W3C Recommendation, January 2020).
+  - `event.version`, `event.group`, `event.count` are removed.
+  - `source.type` is now the source kind (e.g. `browser`, `gtag`, `mcp`, `cli`).
+    New `source.platform` holds the runtime (`web` | `server` | `app` | ...).
+  - `source.id` and `source.previous_id` are removed.
+  - Browser source now sets `source.url` and `source.referrer`.
+  - MCP source sets `source.tool` per emission. CLI source sets
+    `source.command`.
+  - `Entity.nested` and `Entity.context` are now optional. Root `event.nested`
+    and `event.context` remain required.
+  - Each source self-registers via TypeScript module augmentation of `SourceMap`
+    in `@walkeros/core`.
+  - App-side coordination (`/workspaces/developer/app`) is a follow-up plan, not
+    part of this release. Telemetry from v4 CLI/MCP will not validate against
+    the existing app schema until that follow-up ships.
+  - `Mapping.Rule.skip` is renamed to `Mapping.Rule.silent`. Customer flow.json
+    configs using `skip: true` in mapping rules must rename to `silent: true`.
+    Hard cut: no legacy alias, the field is gone.
+
+- 942a7fe: Flow v4: type redesign and cross-flow references.
+
+  Breaking changes:
+  - Renamed `Flow.Settings` (single-flow shape) to `Flow`. The new
+    `Flow.Settings` is the arbitrary kv-bag inside `Flow.Config` (matches
+    `Destination.Settings` semantics).
+  - Renamed `Flow.Config` (root file shape) to `Flow.Json`.
+  - Removed `Flow.Web` and `Flow.Server`. Replaced by
+    `config.platform: 'web' | 'server'` (a string discriminator).
+  - Renamed `Flow.InlineCode` to `Flow.Code`.
+  - Renamed `Flow.SourceReference` / `DestinationReference` /
+    `TransformerReference` / `StoreReference` to `Flow.Source` / `Destination` /
+    `Transformer` / `Store` (Reference suffix dropped).
+  - Renamed `Flow.ContractEntry` to `Flow.ContractRule`.
+  - Lifted `bundle` and platform fields into the per-flow `config` block.
+  - `flow.json` `version` bumped from 3 to 4. v3 input is rejected (no compat
+    shim).
+
+  New:
+  - `$flow.X.Y` reference resolves to `flows.X.config.Y` in the same file.
+    Useful for linking a web flow's API destination to a server flow's deployed
+    URL without duplicating values.
+  - Per-flow `Flow.Config` block: `{ platform, url, settings, bundle }`.
+  - `walkeros validate` warns on unresolved `$flow.X.Y` (use `--strict` to
+    error). `walkeros bundle` and `walkeros deploy` always error on unresolved
+    refs.
+  - See `docs/migrating/v3-to-v4.mdx` on the website for the manual migration
+    steps. No automated codemod is shipped.
+
+- cfc7469: **Breaking — `@walkeros/core`:** `fetchPackage(name, { baseUrl })`
+  now expects the host app to expose the v2 `/api/packages/[name]` endpoint that
+  returns the merged `WalkerOSPackage` shape directly (single round-trip,
+  `?expand=all`). The previous two-fetch pattern (`?path=package.json` +
+  `?path=dist/walkerOS.json`) is removed. Hosts must serve the v2 shape; the
+  offline jsdelivr fallback is unchanged.
+
+  **Feature — CLI/MCP/explorer:** Outbound walkerOS-aware HTTP clients now
+  identify themselves to the configured app origin via
+  `X-Walkeros-Client: walkeros-{cli|mcp}/{version}`. `@walkeros/explorer`
+  exports `setPackageTypesBaseUrl(url?)` so host apps can proxy `.d.ts` through
+  their own origin (used by the walkerOS Tag Manager app to drop the jsdelivr
+  CDN allowance entirely).
+
+- 1ef33d9: **BREAKING:** Unified callback signatures across mapping and on.\*
+  subscriptions.
+
+  Every callback in walkerOS now reads `(data, context) => result`. Sources,
+  transformers, destinations, and stores already conformed; mapping and on.\*
+  join the family in v4.1.
+
+  ### Mapping callbacks
+
+  `fn`, `condition`, and `validate` now share a single shape:
+
+  `(value, context: Mapping.Context) => result`
+
+  `Mapping.Options` is removed. Replaced by `Mapping.Context`:
+
+  ```ts
+  interface Context {
+    event: WalkerOS.DeepPartialEvent;
+    mapping: Value | Rule;
+    collector: Collector.Instance; // required
+    logger: Logger.Instance; // required
+    consent?: WalkerOS.Consent; // resolved consent
+  }
+  ```
+
+  Rule-level `condition` is now `(event, context) => boolean`.
+  `Mapping.Options.props` is removed (no production callers).
+
+  #### Mapping upgrade
+
+  ```ts
+  // before
+  const fn: Mapping.Fn = (value, mapping, options) => /* … */;
+  const cond: Mapping.Condition = (value, mapping, collector) => /* … */;
+  const val: Mapping.Validate = (value) => /* … */;
+
+  // after
+  const fn: Mapping.Fn = (value, context) => /* … */;
+  const cond: Mapping.Condition = (value, context) => /* … */;
+  const val: Mapping.Validate = (value, context) => /* … */;
+  ```
+
+  In `$code:` strings (flow.json):
+
+  ```json
+  // before
+  "fn": "$code:(value, mapping, options) => …"
+  "condition": "$code:(value, mapping, collector) => …"
+
+  // after
+  "fn": "$code:(value, context) => …"
+  "condition": "$code:(value, context) => …"
+  ```
+
+  `context.mapping` replaces the second positional arg; `context.collector`,
+  `context.logger`, and `context.consent` are all available.
+
+  One-arg callbacks like `(value) => value.toUpperCase()` continue to work
+  unchanged.
+
+  ### On.\* subscription callbacks
+
+  `walker.on('consent', …)`, `walker.on('ready', …)`, etc. now receive
+  `(data, context: On.Context) => void | Promise<void>`. The legacy `Context`
+  interface, `*Config` aliases, and `Options` discriminated union are removed.
+
+  ```ts
+  interface Context {
+    collector: Collector.Instance; // required
+    logger: Logger.Instance; // required
+  }
+
+  type Fn<TData = unknown> = (
+    data: TData,
+    context: Context,
+  ) => void | Promise<void>;
+
+  type ConsentFn = Fn<WalkerOS.Consent>;
+  type SessionFn = Fn<Collector.SessionData | undefined>;
+  type UserFn = Fn<WalkerOS.User>;
+  type ReadyFn = Fn<void>;
+  type RunFn = Fn<void>;
+  type GenericFn = Fn<unknown>;
+  ```
+
+  The new `On.Subscription` alias is the registerable union for
+  `walker.on(action, X)`.
+
+  #### On.\* upgrade
+
+  ```ts
+  // before
+  walker.on('consent', { marketing: (collector, consent) => /* … */ });
+  walker.on('ready', (collector) => /* … */);
+  walker.on('session', (collector, session) => /* … */);
+
+  // after
+  walker.on('consent', { marketing: (consent, ctx) => /* … */ });
+  walker.on('ready', (_, ctx) => /* … */);
+  walker.on('session', (session, ctx) => /* … */);
+  ```
+
+  `ctx.collector` replaces the positional first arg; `ctx.logger` is also
+  available.
+
+  ### Why both at once
+
+  Both refactors follow the same `(data, context)` pattern. Shipping them in one
+  release means consumers do one search-and-replace pass instead of two, and the
+  codebase reaches full callback-signature consistency in v4.1.
+
+### Minor Changes
+
+- 8e06b1f: **BREAKING:** Unified reference syntax: `$store:id` and
+  `$secret:NAME` now use the dot separator: `$store.id` and `$secret.NAME`.
+
+  The coherent rule across every walkerOS reference is:
+  - **`.`** key or path (resolver looks up or walks what follows)
+  - **`:`** literal value or raw-code payload (resolver uses what follows
+    verbatim)
+
+  `$var.`, `$def.`, `$env.NAME[:default]`, `$contract.`, and `$code:(…)` are
+  unchanged, they already fit the rule.
+
+  Every shipped example, published `walkerOS.json` metadata, doc page, and skill
+  has been updated. A new canonical reference-syntax guide lives at
+  `/docs/guides/reference-syntax`. Regex constants (`REF_VAR`, `REF_DEF`,
+  `REF_ENV`, `REF_CONTRACT`, `REF_STORE`, `REF_SECRET`, `REF_CODE_PREFIX`) are
+  exported from `@walkeros/core` import these instead of hand-rolling regexes.
+
+  ### Upgrade
+
+  Search-and-replace across your flow configs:
+
+  ```
+  $store:<id>      → $store.<id>
+  $secret:<NAME>   → $secret.<NAME>
+  ```
+
+  Everything else stays the same. Your `$var.*`, `$def.*`, `$env.*`,
+  `$contract.*`, and `$code:*` references need no changes.
+
+### Patch Changes
+
+- 465775c: Add Monaco IntelliSense for `$flow.X` cross-flow references in
+  `Code`/`CodeBox`. Completion offers known sibling flow names from the parsed
+  flow document, hover describes the resolved target, decorations style matches
+  the other reference prefixes, and unknown flow names emit a warning marker.
+  Re-export `REF_FLOW` from `@walkeros/core` so consumers can build inline regex
+  tooling without reaching into the subpath.
+- 3d50dd6: JSON Schema exports now include canonical `id` + `title` (e.g.
+  `Destination.Config`, `Logger.Config`) — replaces anonymous `__schema0` /
+  `object` / `any` labels. Extracts shared `LoggerConfigSchema` (was inlined
+  5×). Removes deprecated `schemas/value-config.ts`. No TypeScript surface
+  changes.
+
+## 3.4.2
+
+## 3.4.1
+
+### Patch Changes
+
+- 12adf24: Step examples can now carry a `title`, `description`, and `public`
+  flag. Non-public examples stay hidden from the docs and AI tools so first-time
+  visitors see only the canonical ones.
+- 75aa26b: `useHooks` now isolates hook failures. A pre-hook that throws no
+  longer crashes the pipeline — the wrapped function is called directly and a
+  warning is logged. A post-hook that throws leaves the original result in
+  place. Added optional 4th `logger` parameter so warnings route through the
+  walkerOS Logger (falls back to `console.warn` when no logger is provided). All
+  collector call sites now pass `collector.logger`.
+
+## 3.4.0
+
+### Minor Changes
+
+- 525f5d9: Introduce the standardized `StepExample.out` shape:
+  `[callable, ...args][]` where each tuple is a function call (first element is
+  the callable name) or a `['return', value]` tuple for transformer-style
+  returns. Every effect is self-describing; docs and tools can render it
+  uniformly without a per-package registry.
+
+  Ship the shared `formatOut` renderer from `@walkeros/core` for docs + app.
+  Also exports `StepEffect` and `StepOut` types. Migrate
+  `@walkeros/web-destination-gtag` to the new shape as the canary — its
+  multi-tool outputs (GA4 + Ads + GTM) now flatten into a single array of
+  `gtag(...)` and `dataLayer.push(...)` tuples in observed execution order.
+  Remaining destination packages ship the old shape until the bulk migration
+  (separate plan).
+
+### Patch Changes
+
+- 74940cc: Add `TransformerSchemas` and `StoreSchemas` namespaces with
+  `ConfigSchema` / `configJsonSchema` exports. Mirrors the existing
+  `DestinationSchemas` / `SourceSchemas` pattern so every component type has a
+  documented Config schema available via `@walkeros/core/dev`.
+
+  Reconcile pre-existing drift between TS Config interfaces and their Zod
+  schemas:
+  - `DestinationSchemas.ConfigSchema` now includes `before`, `next`, `cache`,
+    `disabled`, `mock`, `include` (matching `Destination.Config`) and drops
+    phantom `onError` / `onLog` fields that were never wired in the TS type or
+    consumed at runtime.
+  - `SourceSchemas.ConfigSchema` now includes `disabled` and drops a phantom
+    `onError` field.
+  - `MappingSchemas.ConfigSchema` now includes `include`, matching
+    `Mapping.Config` (the base `Source.Config` extends).
+  - `CollectorSchemas.ConfigSchema` now includes `logger` and drops phantom
+    `verbose` / `onError` / `onLog` fields not present in `Collector.Config`.
+  - `CollectorSchemas.InitConfigSchema` now includes `transformers`, `stores`,
+    `hooks`, matching `Collector.InitConfig`.
+
+  Add a compile-time drift guard at
+  `packages/core/src/schemas/__tests__/config-drift.test-d.ts`. Any future
+  divergence between a Config TS interface and its Zod schema fails
+  `tsc --noEmit` (already wired into the project's `typecheck` script and CI).
+  Keys-only check; value types may still differ for recursive or generic-slot
+  fields where Zod cannot express TS-side precision.
+
+## 3.3.1
+
+## 3.3.0
+
+### Minor Changes
+
+- 2849acb: **BREAKING CHANGE:** The `packages` block has moved from
+  `flow.<name>.packages` to `flow.<name>.bundle.packages`. Flow files using the
+  old shape fail fast with a migration error pointing to the new location.
+
+  Also adds `flow.<name>.bundle.overrides` — a `Record<string, string>` for
+  pinning transitive dependency versions, matching npm's `overrides` semantics.
+  Use this to resolve version conflicts when a transitive dependency's declared
+  range conflicts with another required version in the same tree (the original
+  motivating case: `@amplitude/engagement-browser` pins
+  `@amplitude/analytics-types@^1.0.0` while `@amplitude/analytics-browser`
+  transitively requires `analytics-types@2.11.1` exact — previously an
+  unresolvable bundler conflict).
+
+  **Migration:** move the existing `packages` block one level deeper into a new
+  `bundle` wrapper.
+
+  ```diff
+    {
+      "version": 3,
+      "flows": {
+        "default": {
+          "web": {},
+  -       "packages": {
+  -         "@walkeros/collector": {}
+  -       },
+  +       "bundle": {
+  +         "packages": {
+  +           "@walkeros/collector": {}
+  +         }
+  +       },
+          "sources": { },
+          "destinations": { }
+        }
+      }
+    }
+  ```
+
+  **Overrides example:**
+
+  ```json
+  {
+    "flows": {
+      "default": {
+        "web": {},
+        "bundle": {
+          "packages": {
+            "@walkeros/web-destination-amplitude": {}
+          },
+          "overrides": {
+            "@amplitude/analytics-types": "2.11.1"
+          }
+        }
+      }
+    }
+  }
+  ```
+
+  Overrides only substitute **transitive** dependencies during resolution —
+  direct package specs declared in `bundle.packages` always win. Overrides
+  targeting a direct local-path package emit a warning and are ignored. Peer
+  constraint mismatches against the chosen override emit a warning but do not
+  error (the override is an explicit user directive).
+
+- 08c365a: Add `include` as a first-class field on `Destination.Config`
+  (destination-level) and `Mapping.Rule` (per-event override). The collector
+  resolves `include` in `processEventMapping` before calling `push()`,
+  flattening specified event sections into prefixed key-value pairs (e.g.
+  `data_price: 420`) and merging them as the bottom layer of `context.data`.
+
+  Rule-level `include` replaces config-level (not additive). Merge priority:
+  include (bottom) → config.data → rule.data (top, wins on conflict). The
+  `context` section correctly extracts `[0]` from OrderedProperties tuples.
+
+  New export: `flattenIncludeSections(event, sections)` from `@walkeros/core`.
+
+- 08c365a: Add `skip?: boolean` to `Mapping.Rule` as a universal sibling of
+  `ignore`. Destinations can now honor a rule-level `skip` to process
+  `settings.*` side effects (identify, revenue, group, etc.) while omitting
+  their default forwarding call (`track()`, `capture()`, `event()`). Replaces
+  destination-specific `settings.skipTrack` / `settings.skipEvent` toggles.
+
+  `processEventMapping()` now returns an explicit `skip: boolean` field
+  alongside `ignore`. The collector does not short-circuit on `skip` — it still
+  calls `destination.push()` so the destination can run its side effects. The
+  destination implementation reads `context.rule?.skip` and gates its default
+  forwarding call on `!skip`.
+
+  `ignore: true` still wins when both flags are set on the same rule.
+
+- 08c365a: Add `command` field to `Flow.StepExample` for routing non-event
+  inputs through walker commands (`consent`, `user`, `run`, `config`). Replaces
+  the gtag-only `_consent: true` magic marker pattern. Test runners can now
+  explicitly opt into `elb('walker <command>', in)` instead of pushing `in` as a
+  regular event.
+
+  **Breaking for anyone copying gtag's step-example test runner:** the
+  `_consent: true` magic marker on `mapping` is gone. Migrate to
+  `command: 'consent'` on `Flow.StepExample`.
+
+### Patch Changes
+
+- 08c365a: Expand `getMarketingParameters` to recognise 25+ ad platform click
+  IDs (Pinterest, Reddit, Quora, Yandex, Outbrain, Taboola, Mailchimp, Klaviyo,
+  HubSpot, Adobe, Impact, CJ, Branch, plus Google's `wbraid`/`gbraid`). Add a
+  new `platform` field that resolves the click ID to a canonical platform
+  identifier (e.g. `gclid` → `google`, `fbclid` → `meta`). Multi-click-ID URLs
+  are resolved deterministically via a priority order.
+
+  Custom click-ID registries can be passed as the third argument to
+  `getMarketingParameters`, or via the new `clickIds` field in the session
+  source settings — so flow.json users can extend or override defaults without
+  touching code.
+
+## 3.2.0
+
+### Minor Changes
+
+- eb865e1: Add chainPath to ingest metadata and support path-specific mocks via
+  --mock destination.ga4.before.redact='...'
+- c0a53f9: Flow graph architecture: symmetric before/next hooks, mutable Ingest,
+  per-destination isolation.
+  - Add symmetric `before`/`next` to all step types (sources, transformers,
+    destinations)
+  - Add `Ingest` interface with mutable `_meta` tracking (hops, path)
+  - Parameterize `Transformer.Fn<T, E>` and `Result<E>` on event type
+  - Support `Result[]` return from transformers for fan-out
+  - Remove `Object.freeze(ingest)` — ingest is fully mutable
+  - Upgrade `setIngest` to create typed `Ingest` with `_meta`
+  - Clone ingest per destination to prevent cross-contamination
+  - Add `createMockContext` test utility for context construction
+
+- f007c9f: Wire initConfig.hooks into collector instance. Simulation uses
+  prePush/postDestinationPush hooks for event capture. Hooks are wired by
+  startFlow before events fire.
+- bf2dc5b: Add conditional routing and native cache as built-in config
+  properties on sources, transformers, and destinations.
+
+  **Routing:** NextRule[] in next/before properties enables conditional step
+  chaining, replacing @walkeros/transformer-router.
+
+  **Cache:**
+  - Cache rules use the same match syntax as routing (MatchExpression)
+  - Source cache: full pipeline caching with respond interception
+  - Transformer cache: step-level memoization, chain continues
+  - Destination cache: event deduplication
+  - Update rules modify cached results on read via getMappingValue
+  - Default per-collector memory store with namespaced keys
+
+  compileMatcher upgraded to use getByPath for scoped dot-paths (ingest.method,
+  event.name). Removed @walkeros/server-transformer-cache (replaced by native
+  cache).
+
+- da0b640: Add include/exclude destination filter to collector.push PushOptions.
+  Sources can now control which destinations receive their events. Destination
+  simulation uses the full collector pipeline with include filter, giving
+  production-identical event enrichment, consent, and mapping.
+
+## 3.1.1
+
+## 3.1.0
+
+### Minor Changes
+
+- 966342b: Add mergeConfigSchema to core and integrate into MCP package_get
+  tool.
+
+  package_get now returns schemas.config — a merged JSON Schema combining base
+  config fields (require, consent, logger, mapping, etc.) from core with the
+  package's typed settings schema. Runtime-only fields (env, onError, onLog) are
+  excluded.
+
+- df990d4: Unified source simulation input. All source simulation uses
+  SourceInput { content, trigger?, env? } — one format for CLI, MCP, and tests.
+  Removes legacy runSourceLegacy and deprecated SimulateSource fields. CLI gains
+  --step flag. MCP flow_simulate drops example parameter (use flow_examples to
+  discover, then provide event). flow_examples now returns trigger metadata.
+  StepExample Zod schema aligned with TypeScript type.
+
+### Patch Changes
+
+- dfc6738: MCP api tool: replace overloaded `id` param with explicit `projectId`
+  and `flowId`. CLI functions now throw structured ApiError with code and
+  details from the API response. mcpError forwards structured error data to MCP
+  clients.
+- bee8ba7: Replace hardcoded package registry with live npm search. Package
+  catalog is now fetched dynamically from npm and enriched with walkerOS.json
+  metadata from CDN.
+
+  Change platform type from string to array. Packages declare platform as
+  ["web"], ["server"], or ["web", "server"]. Empty array means
+  platform-agnostic. The normalizePlatform utility handles backwards
+  compatibility with the old string format from already-published packages.
+
+  Remove outputSchema from package_get to prevent SDK validation crashes on
+  unexpected field values.
+
+- 966342b: Add missing fields to Zod schemas: require, logger, ingest on
+  Source.Config and require, logger on Destination.Config. These fields existed
+  in TypeScript types but were absent from schemas, making them undiscoverable
+  via JSON Schema and MCP.
+
+## 3.0.2
+
+## 3.0.1
+
+## 3.0.0
+
+### Major Changes
+
+- 0e5eede: BREAKING: Flow configs now require `"version": 3`. Versions 1 and 2
+  are no longer accepted. To migrate, change `"version": 1` or `"version": 2` to
+  `"version": 3` in your walkeros.config.json.
+- d11f574: Rename Flow.Setup to Flow.Config and Flow.Config to Flow.Settings for
+  consistent Config/Settings naming convention at every level. Breaking change:
+  all type names, function names, schema names, and API URL paths (/configs →
+  /settings) updated.
+- 23f218a: Replace flat/v2 contract format with named contracts supporting
+  extends inheritance.
+
+  BREAKING CHANGES:
+  - `contract` is now a map of named contract entries (e.g.,
+    `{ "default": { ... }, "web": { ... } }`)
+  - `version` field inside contracts removed
+  - `$tagging` renamed to `tagging`
+  - Legacy flat contract format removed
+  - `$globals`, `$context`, `$custom`, `$user`, `$consent` references removed
+  - Settings-level `contract` field removed (use named contracts at config
+    level)
+  - Auto-injection of `$tagging` into `collector.tagging` removed (use
+    `$contract.name.tagging` explicitly)
+  - Validator `contract` setting renamed to `events` (receives raw schemas, not
+    `{ schema: ... }` wrappers)
+
+  NEW FEATURES:
+  - Named contracts with `extends` for inheritance (additive merge)
+  - Generalized dot-path resolution: `$def.name.nested.path`,
+    `$contract.name.section`
+  - `$contract` as first-class reference type with path access
+  - `$def` inside contracts supported via two-pass resolution
+  - `$def` aliasing for reducing repetition: `{ "c": "$contract.web" }` then
+    `$def.c.events`
+
+### Minor Changes
+
+- 6ae0ee3: Add v2 structured contract format with globals, context, custom,
+  user, and consent sections.
+
+  Contracts can now describe cross-event properties (globals, consent, etc.)
+  alongside entity-action event schemas. Top-level sections are JSON Schemas
+  that merge additively into per-event validation.
+
+  Breaking: None. Legacy flat contracts continue working unchanged. v2 is opt-in
+  via `version: 2` field.
+
+- 1fe337a: Add hints field to walkerOS.json for lightweight AI-consumable
+  package context.
+
+  Packages can now export a `hints` record from `src/dev.ts` containing short
+  actionable tips with optional code snippets. Hints are serialized into
+  `walkerOS.json` by buildDev() and surfaced via the MCP `package_get` tool.
+
+  Pilot: BigQuery destination includes hints for authentication, table setup,
+  and querying.
+
+- c83d909: Add Store types as fourth modular component type. Stores provide
+  pluggable key-value storage (get/set/delete/destroy) with sync and async
+  support for browser and server backends.
+- b6c8fa8: Add stores as a first-class component type in Flow.Config. Stores get
+  their own `stores` section in flow settings, a `collector.stores` registry,
+  and `$store:storeId` env wiring in the bundler. Includes `storeMemoryInit` for
+  Flow.Config compatibility and type widening in cache/file transformers.
+
+### Patch Changes
+
+- 2b259b6: Fix deterministic package version resolution in bundler.
+  - Two-phase resolve-then-install prevents version overwrites
+  - peerDependencies resolved at lowest priority (not equal to deps)
+  - Per-build temp directories prevent cross-build interference
+  - Optional peerDeps (peerDependenciesMeta) correctly skipped
+  - Prerelease versions handled with includePrerelease flag
+  - Package names validated against npm naming rules
+
+- 2614014: Fix: `consent: {}` on destination config now auto-grants instead of
+  blocking all events forever
+- 37299a9: Extract match logic (compileMatcher, MatchExpression, MatchCondition,
+  MatchOperator, CompiledMatcher) from router to core as shared utility. Router
+  now imports from core — no public API changes.
+- 499e27a: Fix getByPath breaking on falsy intermediate values (0, false, "")
+- d11f574: Fix $var/$def/$env resolution in transformer configs and env fields
+
+  Previously, `resolvePatterns` was not called on transformer configs or any
+  component's `env` field. This meant `$var.name`, `$def.name`, and `$env.NAME`
+  references in those positions were passed through as literal strings. Now all
+  component types (sources, destinations, transformers, stores) have both
+  `config` and `env` resolved consistently.
+
+- 5cb84c1: Replace hand-written MCP resources with auto-generated JSON Schemas
+  from @walkeros/core. Add walkerOS.json to 5 transformer packages. Variables
+  resource remains hand-maintained (runtime interpolation patterns).
+- 499e27a: Add sideEffects declarations to all packages for bundler tree-shaking
+  support.
+
+## 2.1.1
+
+### Patch Changes
+
+- fab477d: Replace union transformer return type with unified
+  `Transformer.Result` object. Transformers now return `{ event }` instead of
+  naked events, and can optionally include `respond` (for wrapping) or `next`
+  (for branching). The `BranchResult` type and `__branch` discriminant are
+  removed.
+
+## 2.1.0
+
+### Minor Changes
+
+- 7fc4cee: Add default values and improved descriptions to flow schemas for
+  better IDE IntelliSense
+- cb2da05: Add data contracts for centralized event validation and documentation
+- 2bbe8c8: Add destroy lifecycle method to all step types (sources,
+  destinations, transformers) and shutdown command to collector
+- 3eb6416: Add unified `env.respond` capability. Any step (transformer,
+  destination) can now customize HTTP responses via
+  `env.respond({ body, status?, headers? })`. Sources configure the response
+  handler — Express source uses createRespond for idempotent first-call-wins
+  semantics. CLI serve mode removed (superseded by response-capable flows).
+- 02a7958: Add WARN log level (ERROR=0, WARN=1, INFO=2, DEBUG=3). Logger
+  instances expose `warn()` method routed to `console.warn` and `json()` method
+  for structured output. Config accepts optional `jsonHandler`. MockLogger
+  includes both as jest mocks. CLI logger unified with core logger via
+  `createCLILogger()` factory.
+- 026c412: Unified simulation API: single simulate() function replaces
+  simulateSource/simulateDestination/simulateTransformer/simulateFlow. Built-in
+  call tracking for destinations via wrapEnv. No bundling required for
+  simulation.
+- 7d38d9d: Add `validateFlowSetup` for portable Flow.Setup validation with
+  line/column positions and IntelliSense context extraction.
+
+### Patch Changes
+
+- 7fc4cee: Add contract as optional property to Flow.Setup schema
+- 97df0b2: Step examples: upgrade all packages to blueprint pattern with inline
+  mapping, no intermediate variables, no `all` export
+- 97df0b2: Step examples: add mapping field to StepExample type, rewrite Meta
+  Pixel examples with functional tests
+
+## 2.0.1
+
+### Patch Changes
+
+- e34c11e: Align all packages to unified v2 with consistent dependency structure
+
+## 1.4.0
+
+### Minor Changes
+
+- 7b2d750: Add walkerOS.json package convention for CDN-based schema discovery
+
+## 1.3.0
+
+### Minor Changes
+
+- a4cc1ea: Add collector.status for per-source and per-destination delivery
+  tracking
+
+## 1.2.2
+
+### Patch Changes
+
+- 7ad6cfb: Fix transformer chains computed on-demand instead of pre-computed
+
+  Transformer chains configured via `destination.before` now work correctly.
+  Previously, chains were pre-computed at initialization but the resolution
+  function was never called, causing `before` configuration to be silently
+  ignored.
+
+  **What changed:**
+  - Chains now compute at push time from `destination.config.before`
+  - Removed unused `collector.transformerChain` state
+  - Removed dead `resolveTransformerGraph()` function
+  - Dynamic destinations now support `before` property
+
+## 1.2.1
+
+### Patch Changes
+
+- 6256c12: Add inline code support for sources, transformers, and destinations
+  - Add `InlineCodeSchema` with `push`, `type`, and `init` fields for embedding
+    JavaScript in flow configs
+  - Make `package` field optional in reference schemas (either `package` or
+    `code` required at runtime)
+  - Update `flow-complete.json` example with inline code demonstrations
+    including enricher transformer, debug destination, and conditional mappings
+
+## 1.2.0
+
+### Minor Changes
+
+- f39d9fb: Add array support for transformer chain configuration
+
+  Enables explicit control over transformer chain order by accepting arrays for
+  `next` and `before` properties, bypassing automatic chain resolution.
+
+  **Array chain behavior:**
+
+  | Syntax                           | Behavior                                               |
+  | -------------------------------- | ------------------------------------------------------ |
+  | `"next": "validate"`             | Walks chain via each transformer's `next` property     |
+  | `"next": ["validate", "enrich"]` | Uses exact order specified, ignores transformer `next` |
+
+  **Example:**
+
+  ```json
+  {
+    "sources": {
+      "http": {
+        "package": "@walkeros/server-source-express",
+        "next": ["validate", "enrich", "redact"]
+      }
+    },
+    "destinations": {
+      "analytics": {
+        "package": "@walkeros/server-destination-gcp",
+        "before": ["format", "anonymize"]
+      }
+    }
+  }
+  ```
+
+  When walking a chain encounters an array `next`, it appends all items and
+  stops (does not recursively resolve those transformers' `next` properties).
+
+- 888bbdf: Add inline code syntax for sources, transformers, and destinations
+
+  Enables defining custom logic directly in flow.json using `code` objects
+  instead of requiring external packages. This is ideal for simple one-liner
+  transformations.
+
+  **Example:**
+
+  ```json
+  {
+    "transformers": {
+      "enrich": {
+        "code": {
+          "push": "$code:(event) => ({ ...event, data: { ...event.data, enriched: true } })"
+        },
+        "config": {}
+      }
+    }
+  }
+  ```
+
+  **Code object properties:**
+  - `push` - The push function with `$code:` prefix (required)
+  - `type` - Optional instance type identifier
+  - `init` - Optional init function with `$code:` prefix
+
+  **Rules:**
+  - Use `package` OR `code`, never both (CLI validates this)
+  - `config` stays separate from `code`
+  - `$code:` prefix outputs raw JavaScript at bundle time
+
+## 1.1.0
+
+### Minor Changes
+
+- 20eca6e: Breaking change: Unified dynamic pattern syntax in Flow
+  configuration, sorry!
+
+  **New syntax:**
+  - `$def.name` - Reference definitions (replaces
+    `{ "$ref": "#/definitions/name" }`)
+  - `$var.name` - Reference variables (replaces `$variables.name`)
+  - `$env.NAME` or `$env.NAME:default` - Reference environment variables
+
+  **Migration:**
+
+  | Old Syntax                              | New Syntax                             |
+  | --------------------------------------- | -------------------------------------- |
+  | `{ "$ref": "#/definitions/itemsLoop" }` | `$def.itemsLoop`                       |
+  | `$variables.currency`                   | `$var.currency`                        |
+  | `${GA4_ID}` or `${GA4_ID:default}`      | `$env.GA4_ID` or `$env.GA4_ID:default` |
+
+  **Note:** Only `$env` supports defaults (`:default`) because environment
+  variables are external and unpredictable. Variables (`$var`) are explicitly
+  defined in config, so missing ones indicate a configuration error and will
+  throw.
+
+  **Example:**
+
+  ```json
+  {
+    "variables": { "currency": "EUR" },
+    "definitions": {
+      "itemsLoop": { "loop": ["nested", { "map": { "item_id": "data.id" } }] }
+    },
+    "destinations": {
+      "ga4": {
+        "config": {
+          "measurementId": "$env.GA4_ID:G-DEMO123",
+          "currency": "$var.currency",
+          "items": "$def.itemsLoop"
+        }
+      }
+    }
+  }
+  ```
+
+### Patch Changes
+
+- b65b773: Queue on() events until destination init completes
+
+  Destinations now receive `on('consent')` and other lifecycle events only after
+  `init()` has completed. Previously, `on()` was called before `init()`,
+  requiring workarounds like gtag's `initializeGtag()` call inside its `on()`
+  handler.
+
+  Also renamed queue properties for clarity:
+  - `destination.queue` → `destination.queuePush`
+  - `destination.onQueue` → `destination.queueOn`
+
+## 1.0.0
+
+### Major Changes
+
+- 67c9e1d: Hello World! walkerOS v1.0.0
+
+  Open-source event data collection. Collect event data for digital analytics in
+  a unified and privacy-centric way.