@fabricorg/platform-host 8.0.0 → 8.1.0

package/CHANGELOG.md

@@ -1,5 +1,72 @@
 # @fabricorg/platform-host
 
+## 8.1.0
+
+### Minor Changes
+
+- e81defa: Add `eventAppendChecks()`, `runEventAppendChecks(subject)`, and their public subject, check,
+  and result types for custom store authors. Six checks exercise first-write-wins event appends
+  through plain, outbox and transaction paths, including buffered and committed duplicates and
+  plain-to-outbox behavior. Each check uses earlier and later duplicates with changed envelope and
+  metadata fields, including added map keys. Whole-record comparisons normalize key order, dates
+  and nulls; numeric strings are normalised only for `sequence`, `attemptCount` and `leaseToken`.
+  The first returned records supply snapshot references, allowing custom envelope fields and fields
+  the store omits or returns as null, including required fields. Transaction checks verify after commit
+  without requiring read-your-writes. Each committed-duplicate transaction also appends a fresh plain
+  control event after the duplicate and verifies its presence after commit to detect a silently
+  rolled-back unit of work.
+  Outbox snapshots cover pending, probe-leased, retry-pending and published rows, including lease
+  fields. Retry-pending accepts cleared or expired leases or the probe owner retained without an
+  expiry; a probe lease still live more than five seconds past the probe's clock reports a failure
+  to release it, and an expiry inside that margin is waited out as clock skew. Attempt counts and error formatting are
+  not prescribed. Retry and retirement availability use whole seconds a few seconds in the past,
+  tolerating coarse timestamp precision and small clock skew. Lease tokens come from claims, not
+  optional list views.
+  Absence checks require a fresh control outbox row.
+
+  These checks detect common first-write-wins violations: throwing, replacing or merging events,
+  keeping both events, selecting the earlier-ordered or later-ordered duplicate, adding outbox rows,
+  and resetting or rewriting an existing outbox row in the probed states. They are not exhaustive.
+  Known limits are rules comparing ordering fields in mixed directions, store-wide uniqueness across
+  spaces or tenants, the dead-letter state, and claims leasing foreign rows without returning them.
+  The checks also cannot detect a buffering transaction that merges fields the store never returns
+  (such as `includeProvenance`) from a buffered duplicate, or duplicate handling backed by an
+  in-process cache that works within one run but not across restarts. Stores requiring gapless
+  sequences, or unique keys over fields the control events share with the probe event, can fail
+  falsely.
+
+  Missing append capabilities skip; missing outbox verification or cleanup methods (including
+  `markOutboxPublished`) fail. Run in test or staging with outbox relays paused for the probe tenant.
+  Event timestamps use whole seconds slightly in the past; probes leave append-only events in dedicated
+  `event-append-probe` spaces. Successful outbox-creating checks mark the row published without
+  sending anything; final cleanup attempts to dead-letter visible pending control/failure rows.
+  Published exercises duplicate appends against that delivery state. Absence-control rows remain
+  dead-lettered; operator requeue tooling could publish them later, so exclude probe spaces from real
+  publication and requeue workflows.
+
+  Claims use real current time, a pending-probe-row limit and a 30-second lease. A filter-ignoring
+  claim can still lease foreign rows and make its own writes, such as incrementing attempts. The
+  checks write nothing to foreign rows and stop claiming for that run; later outbox checks fail as
+  skipped because of the earlier filter violation. Standalone checks stop claiming for that check.
+  After a filter violation, cleanup dead-letters only probe rows returned by that claim, using the
+  probe worker and claim tokens without further claims. The failure lists visible probe rows still
+  pending and instructs manual dead-lettering, including unreturned rows or failed retirements.
+  Leases lapse within the requested duration if the store honours it. Visible relay interference
+  fails with instructions to pause relays. These are additive exports; Host and shipped-store runtime behavior is unchanged.
+
+### Patch Changes
+
+- 1effb28: Document first-write-wins event appends for custom store authors. Host re-appends lifecycle events under stable IDs on retry and recovery, with a fresh sequence and timestamps and sometimes a different payload, so `appendEvent` and `appendEventWithOutbox` must treat an event ID the store already holds as a no-op that keeps the first event, without throwing, replacing or merging it. A store that refuses these appends can leave recovery unable to converge; custom stores that throw on a conflicting append need to change. The shipped memory and PostgreSQL stores already conform and enforce ID uniqueness across the whole store, not per tenant or space. `appendEventWithOutbox` for an ID already stored adds no outbox row, even when the first append had none, so `outbox.shouldPublish` should be deterministic for a given event ID. No runtime behavior changes.
+- 5af8a02: Lifecycle record ids no longer collide when their key is cut to length (issue #90).
+
+  Host derives event, attestation, reconciliation and policy-evaluation ids from a key that it sanitises and used to cut to 96 characters. Two keys that differed only after the cut used to get the same id, and since stores keep the first write for an existing id, the later record was dropped silently. The realistic case was a vendor `externalReference` long enough to cut off the outcome, which merged a second `ExternalOperationContradicted` event into the first.
+
+  A key longer than 96 characters after sanitising is now cut to 64 characters followed by `_` and a digest of the whole raw key, 97 characters in all. Being one character over the cap, it cannot equal the id of any key that fits. Ids whose key fits are byte-identical to earlier releases.
+
+  Upgrade note: only ids whose composed key exceeds 96 characters change. Getting there takes a long vendor-supplied value, such as an `externalReference` or external operation id, or an unusually long policy id, vendor, provider, operation name or custom invocation id. An invocation that is in flight across the upgrade and re-appends such a record during retry or recovery writes it under the new id, next to the one written before the upgrade.
+
+  Not changed: keys within the limit that differ only in characters outside `[A-Za-z0-9_-]` (for example policy ids `a.b` and `a/b`) still share an id. Changing that would alter almost every existing id, because host's own keys use `:` and policy ids conventionally use `.`.
+
 ## 8.0.0
 
 ### Major Changes

package/README.md

@@ -255,6 +255,19 @@ the adapter loop. Saga compensation is unchanged. Withheld-output and refused-ha
 reconciliation retain the degraded pattern. Adapter-row fencing and overdue-sweep/late-callback
 rules are described below.
 
+Event appends are first-write-wins by event ID. Host re-appends lifecycle events under stable IDs
+on retry and recovery, and each later attempt carries a fresh `sequence` and timestamps and can carry
+a different payload. A custom store's `appendEvent` and `appendEventWithOutbox` must treat an ID it
+already holds as a no-op that keeps the first event: do not throw, replace, or merge it. A store that
+refuses these appends can leave recovery unable to converge, so the invocation stays `running` until
+the recovery budget settles it, even when the adapter effect succeeded. Custom stores that currently
+throw on a conflicting append need to change. The shipped stores enforce ID uniqueness across the
+whole store, not per tenant or space. The rule covers the outbox too: `appendEventWithOutbox` for an
+ID already stored adds no outbox row, including when the first append went through plain
+`appendEvent`, so that event is never published later. Keep `outbox.shouldPublish` deterministic for
+a given event ID. Custom store authors can verify these paths with the
+[event append conformance checks](#event-append-conformance).
+
 ### Conditional adapter-row writes (8.0.0, issue #79)
 
 `PlatformHostStore.writeAdapterInvocation(input)` is optional. Memory and PostgreSQL implement it;
@@ -1069,6 +1082,88 @@ refusals and an admitted update in our scope. The two adapter isolation checks
 fail for stores that ignore scope. Deployments running `runTenantIsolationChecks` against a custom
 store will see new failures until that store honours the argument.
 
+## Event append conformance
+
+`eventAppendChecks()` and `runEventAppendChecks(subject)` run six checks for first-write-wins
+event IDs: plain appends, outbox appends, outbox-after-plain, transactional appends,
+transactional outbox appends, and transactional outbox-after-plain. Each sends earlier and later
+duplicates with lower and higher sequences, earlier and later occurrence/recording times, and
+different remaining envelope and outbox metadata fields. Transaction checks cover buffered
+duplicates and duplicates against a committed event; they do not require transaction reads to
+expose buffered writes. Each committed-duplicate transaction appends a fresh plain control event
+after the duplicate and verifies its presence after commit, detecting a silently rolled-back unit
+of work. Event-count checks count only the original probe ID; controls use distinct sequences.
+
+The checks detect common first-write-wins violations: throwing, replacing or merging events,
+keeping both events, selecting the earlier-ordered or later-ordered duplicate, adding outbox
+rows, and resetting or rewriting an existing outbox row in the probed states. They are not
+exhaustive. Known limits include rules that compare ordering fields in mixed directions,
+store-wide uniqueness across spaces or tenants, the dead-letter state, and claims that lease
+foreign rows without returning them. They also cannot detect a buffering transaction that merges
+fields the store never returns (such as `includeProvenance`) from a buffered duplicate, or
+duplicate handling backed by an in-process cache that works during one run but fails across
+restarts. Stores requiring gapless sequences can fail falsely: the first event uses sequence 50
+in a fresh space. So can stores with a unique key over fields the control events share with the
+probe event, such as `actionInvocationId` with the subject.
+
+Outbox-creating checks verify the first event and metadata, then compare whole-row snapshots
+while pending, leased by the probe worker, retry-pending, and settled as published. Retry-pending
+accepts a cleared lease, an expired lease with no owner or the probe worker, or the probe owner
+retained as a record without an expiry. A probe lease that still expires more than five seconds
+after the probe's clock fails with `markOutboxFailed did not release the lease`; an expiry inside
+that margin is treated as clock skew and waited out before the row is claimed again. A lease held
+by another worker fails with the request to pause relays. The checks prescribe neither a positive attempt
+count nor error formatting; they snapshot the values the store returns. Retry availability uses
+whole seconds a few seconds in the past, tolerating coarse timestamp precision and small clock
+skew, then the probe reclaims the row before publication. Snapshots include the full envelope,
+lease and publication fields. Comparisons sort object keys, normalize dates and date strings to milliseconds, treat null like undefined, and
+compare numeric strings in `sequence`, `attemptCount` and `leaseToken` as numbers. An absent
+actor disclosure is accepted; included disclosure or actor identity on the first outbox envelope
+fails because the first metadata does not opt in. A store's first returned records supply the
+snapshot references; extra envelope fields and fields the store omits or returns as null are
+accepted, including required fields. This tolerance does not validate envelope completeness. For
+buffered duplicates in the first transaction, exposed fields that differ between the first input
+and a duplicate must retain the first value. Outbox-after-plain checks write a fresh control
+event through the matching outbox seam and require its row to be the only outbox row visible in
+the probe space, so absence verification cannot pass solely because `listOutbox` returns
+nothing.
+
+`EventAppendSubject<TDb>` supplies `store`, `tenantId`, and optional `reset()`. Only
+`appendEvent` and `listEvents(tenantId, spaceId)` are required; the subject does not need a full
+recoverable store. The result has `passed` and per-check `id`, `status`, `evidence`, and
+optional `error`, following the tenant-isolation suite. Absent outbox or transaction append
+capabilities produce `skipped: ` evidence; `passed` means no failures, so inspect skips for
+capability coverage. An available outbox append without `listOutbox`, `claimOutbox`,
+`markOutboxPublished` or `markOutboxFailed` fails because the check cannot verify or clean up.
+Append errors and probe retirement failures also fail.
+
+Run against test or staging stores, preferably with a dedicated probe tenant. **Pause outbox
+relays for the probe tenant.** Event timestamps and written availability values (including
+retirement) use whole seconds a few seconds in the past. Claims and publication markers use
+real current time. Probe rows are expected to be due when availability follows `recordedAt`;
+stores assigning availability by their own clock may need additional setup. Visible foreign-worker
+leases or status changes before the probe acts fail with a request to pause relays. This detects visible interference; it does not prevent a relay race.
+Append-only events remain in fresh `event-append-probe-${uuid}` spaces. Successful
+outbox-creating checks leave their row **published although nothing was sent**; the space name
+identifies probe rows. Published is needed to exercise duplicate appends against that delivery
+state. Absence-control rows and failure rows instead go through final cleanup, which attempts
+to dead-letter visible probe rows still pending with `event append probe; never published`.
+Operator tooling can requeue those dead-lettered rows for real publication: exclude probe
+spaces from real publication and requeue workflows.
+
+Claims use the real current time, a 30-second lease, and a limit equal to the pending probe row
+count. Cleanup filters listed rows locally by tenant and space and writes only to probe rows. If
+a claim returns foreign rows, the failure names their IDs, the checks write nothing to those
+rows, and no further claims occur in that run. Later outbox checks fail with an explanation that
+they were skipped because of the earlier filter violation; a standalone `check.run(subject)`
+stops claims for that check. Cleanup after a filter violation can dead-letter only probe rows
+returned by the violating claim, using the probe worker and that claim's tokens without another
+claim. The failure lists visible probe rows still pending, including unreturned rows or failed
+retirements, and instructs the operator to dead-letter them by hand. A filter-ignoring claim can
+still impose a short lease and make whatever writes the claim itself makes, such as incrementing attempt counts. Those leases lapse
+within 30 seconds of the claim if the store honours the requested duration; the checks do not
+release foreign leases or change foreign statuses, availability or errors.
+
 ## Runtime observability and health
 
 Platform Host exposes vendor-neutral lifecycle telemetry through the optional `telemetry` sink on