@fabricorg/platform-host 7.3.0 → 7.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +80 -0
- package/README.md +236 -15
- package/dist/index.cjs +416 -80
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +135 -17
- package/dist/index.d.ts +135 -17
- package/dist/index.js +413 -79
- package/dist/index.js.map +1 -1
- package/package.json +4 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,85 @@
|
|
|
1
1
|
# @fabricorg/platform-host
|
|
2
2
|
|
|
3
|
+
## 7.5.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- e32ee98: Add opt-in tenant/space/vendor circuit-breaker isolation (issue #52). Platform exports `AdapterCircuitBreakerScope` and `scopedAdapterCircuitBreakerKey` from the root and adapters subpath, accepts optional scope in providers, and adds registry `key` and `getByKey` APIs. Host passes trusted invocation tenant/space scope to providers.
|
|
8
|
+
|
|
9
|
+
The default `adapterType:operation` key and its shared failure domain are unchanged. Consumers can inject `new AdapterCircuitBreakerRegistry({ key: scopedAdapterCircuitBreakerKey })` to prevent one tenant's failures from refusing another tenant's calls. The helper encodes components, distinguishes absent space/vendor from empty values, and uses a separate vendor-specific unscoped namespace when scope is absent. Calls without scope share that namespace; tenant isolation requires scope. Keys appear in telemetry and errors, so use non-secret identifiers.
|
|
10
|
+
|
|
11
|
+
The first config seen for a key wins, including custom keys; later conflicting configs reuse the existing breaker without throwing. Existing one-argument providers remain compatible. Host's `workspace:^` Platform peer declaration is unchanged: passing an extra argument is compatible with older implementations, and Host adds no runtime import of the new helper. Release tooling resolves the workspace peer against the released Platform version.
|
|
12
|
+
|
|
13
|
+
- cfada18: Restrict the idempotent same-state replay shortcut to the invocation that wrote the current recorded state (issue #55).
|
|
14
|
+
|
|
15
|
+
**Release note: validation change.** The shortcut now applies only to a retry of the invocation whose event set the current recorded state. A new invocation re-applying an already-reached state is validated normally and fails unless the state machine defines that edge. Model a self-loop for a new mutation, or submit with the same idempotency key to deduplicate to the original invocation at submission. An idempotent action targeting the machine's initial state on an entity with no recorded state (or a binding with no entity ID), previously admitted by the shortcut, is now validated normally and needs an initial-state self-loop.
|
|
16
|
+
|
|
17
|
+
Stores and mutation transactions can implement the new optional `getEntityStateSource()` method to return the state and invocation writer of the event selected by `getEntityState()`. Memory and PostgreSQL stores provide it, including transaction-scoped reads. Custom stores without the method fall back to subject-scoped `listEvents()` provenance, or to normal transition validation when neither capability is available. Unavailable provenance does not authorize the shortcut.
|
|
18
|
+
|
|
19
|
+
- f136016: Add optional Host redaction hooks for adapter output, error text, and external completion results (issue #53).
|
|
20
|
+
|
|
21
|
+
`redactAdapterOutput` runs before adapter output persistence and handoff processing. Adapter rows, success events and `extractExecutionAttestation` receive redacted output. A throwing hook or null, array, or non-object return withholds output. Host writes its reconciliation verdict before audit evidence and the succeeded adapter marker, retrying a failed idempotent verdict write once while retaining lease fencing. After a durable verdict, direct execution/recovery do not repeat the adapter. Later evidence/marker failures retain the verdict but can leave incomplete evidence; if both verdict-write attempts fail, the adapter can run again on recovery, the same exposure as a crash mid-step. Adapters behind these paths should be idempotent or carry a stable idempotency key. Lease loss can also prevent this branch from recording its verdict. A first accepted reference is retained for callback evidence.
|
|
22
|
+
|
|
23
|
+
`redactErrorText` receives exported `ErrorTextRedactionContext`. Empty error input bypasses the hook unchanged. For non-empty input, execution paths use a fixed placeholder if the hook throws or returns a non-string or empty string. Completion error-text redaction failures instead refuse before writes as non-retryable `redaction_failed`. Circuit-open and consistency messages pass through this hook as adapter text; Host's own fixed text bypasses it.
|
|
24
|
+
|
|
25
|
+
`redactExternalCompletionResult` runs before merge, validation and persistence; a throw or null, array, or non-object return refuses before writes. Both object hooks accept class instances and Dates, using the existing adapter-output `isRecord` predicate. Stored digests hash redacted result/error values when hooks are set. A matching legacy raw incoming digest is acknowledged in memory before invoking hooks; newly stored digests use redacted values. Contradiction detection compares retained values otherwise. Keep redactors deterministic and stable across deploys; a changed redactor can make a repeat look like different evidence. Existing records are not rewritten.
|
|
26
|
+
|
|
27
|
+
Consumers with an exhaustive switch over `ExternalCompletionRefusal` need the new `redaction_failed` case. `redactActionParameters` and `redactAdapterInput` behavior is unchanged. Independently of the new hooks, a worker that loses its lease inside an adapter step now rethrows `RecoverableFinalizationError` without marking the adapter failed, preserving the new owner's row. Refused handoffs write their reconciliation verdict before the adapter marker and emit safe success/accepted audit events. These are default-behavior changes.
|
|
28
|
+
|
|
29
|
+
- 4e429a3: Add `RetryableExecutionError` and `isRetryableExecutionError` to Platform's root and policies exports (issue #56). The error accepts a message and optional cause; its stable name and `EXECUTION_RETRYABLE` code support recognition across package copies.
|
|
30
|
+
|
|
31
|
+
Host adds `retryableExecutionMaxAttempts` (positive integer, default 5). During governance before policy-evaluation persistence, the error is rethrown unchanged for a leased, idempotent invocation below that cap, excluding approval resume. Workers report it through `onError`, stop the heartbeat, and can reclaim the invocation after lease expiry. Cycle results and `worker.cycle.completed` telemetry include an optional `deferred` count when positive; it counts every `RetryableExecutionError` that escapes execution, and those do not increment `failed`.
|
|
32
|
+
|
|
33
|
+
The retry region covers parameter validation, mutation-governance resolution and its single-record checkpoint write, HITL evaluation and its single-record checkpoint write, entitlement, execution authorization, policy resolution, and policy evaluation. Host reuses persisted mutation-governance and HITL decisions through checkpoint guards. The region ends before the first policy-evaluation record is written. Policy persistence, `ComplianceBlocked`, obligation listing/resolution/persistence, state-machine checks, the handler transaction, handlers, adapters, and finalization retain their existing failure/reconciliation/recovery semantics.
|
|
34
|
+
|
|
35
|
+
**Release note: failure semantics (issue #56).** Previously a transient thrown governance error terminally failed the invocation. Opting into `RetryableExecutionError` now leaves an eligible invocation `running` for lease-based recovery, with no failure/status write and no policy evidence written by that attempt. Unleased inline execution, approval resume, non-idempotent actions, and attempts at the cap fail with an explanatory message through the existing redactor. Approval resume's temporary lease does not establish that a recovery worker is configured. Obligation-resolver and policy/obligation-persistence errors remain terminal. Application callbacks should be repeatable and use stable invocation-scoped evidence. Host's Platform peer range remains `workspace:^` so published packages require the Platform release containing the new runtime exports.
|
|
36
|
+
|
|
37
|
+
- 90c1266: Scope adapter invocation reads and writes by tenant, space, and owning action invocation (issue #57). Export `AdapterInvocationScope` and add it as an optional trailing argument to store get/update methods; Host supplies it from the store-loaded invocation, and the shipped memory and PostgreSQL stores enforce all three fields when supplied. Calls without scope retain ID-only behavior for compatibility. Custom store implementers must honour the new optional `scope` argument: the two new adapter isolation checks fail for a store that ignores it, so a deployment running `runTenantIsolationChecks` against a custom store will see new failures until the store scopes these lookups. `TenantIsolationCheck["id"]` gains two members and `runTenantIsolationChecks` results now carry eight entries, so exhaustive switches over check IDs need the additional cases.
|
|
38
|
+
- 93b9830: Host now emits `StateTransitioned` after a validated bound state-machine transition and successful handler execution, following the action's `eventPhase` (issue #54).
|
|
39
|
+
|
|
40
|
+
The version-1 payload contains `entityType`, `entityId`, `fromState`, `toState`, and `causedByAction`. The subject identifies the bound entity. Host appends the fact immediately after the handler's domain events using a stable invocation-scoped event ID. With `transactionWithEvents`, it shares the domain events' transaction in the selected phase; otherwise it follows the legacy non-atomic boundary, as domain events do. It is published through the outbox by default; applications can filter it with `outbox.shouldPublish`.
|
|
41
|
+
|
|
42
|
+
Under `before_adapters`, the transition is appended during handler execution. Under `after_adapters`, it is deferred to finalization after required adapter steps succeed or accept a parked external handoff. Adapter failure withholds both domain events and the transition, so the invocation does not advance Host's recorded entity state even though handler writes may have committed. Atomic finalization commits the deferred events and invocation update together; fallback stores retain non-atomic append and update behavior. Invalid transitions, failed handlers, empty targets, undefined or empty entity IDs, and replay-only acceptance in the target state do not emit a Host transition fact.
|
|
43
|
+
|
|
44
|
+
Applications that append their own `StateTransitioned` for the bound entity get no duplicate from Host: the handler's returned events suppress Host emission when both subject type and ID match, in either event phase. Those applications should drop the workaround to use Host emission.
|
|
45
|
+
|
|
46
|
+
**Release note: validation change.** Previously, when an application did not write `toState` in its events, Host's recorded state remained unset and validation fell back to the initial state on later calls. Host's new transition fact advances recorded state, so later calls validate against that advanced state. A non-idempotent bound action that previously passed repeatedly (for example, `draft -> active`) can now be rejected on the second call (`active -> active`) unless the machine defines that edge. Model the missing edges or self-loops, or submit with the same idempotency key to deduplicate to the original invocation at submission. Applications that already emitted `toState` in their domain events see no change to this validation behavior when that `toState` matches the binding's target state; otherwise Host's fact, appended after the domain events, sets the recorded state to the target. This minor release introduces the stated failure semantics.
|
|
47
|
+
|
|
48
|
+
## 7.4.0
|
|
49
|
+
|
|
50
|
+
### Minor Changes
|
|
51
|
+
|
|
52
|
+
- eac74a9: Adapter circuit breakers now work. `AdapterImplementation.circuitBreaker` was declared and documented, but nothing read it. Platform now implements the breaker, and Host enforces it (issue #22).
|
|
53
|
+
|
|
54
|
+
**Release note: new failure semantics.** An adapter that declares `circuitBreaker: { mode: "enforce", ... }` can now have calls refused. Before this release that config had no effect. Review every adapter that already declares `mode: "enforce"` before upgrading, or switch it to `"monitor"` to observe first.
|
|
55
|
+
|
|
56
|
+
- **`@fabricorg/platform` (adapters subpath and root).**
|
|
57
|
+
- New exports: `InMemoryAdapterCircuitBreaker`, `AdapterCircuitBreakerRegistry`, `AdapterCircuitOpenError`, `isAdapterCircuitOpenError`, `classifyCircuitBreakerOutcome`, `assertAdapterCircuitBreakerConfig`, and the types `AdapterCircuitBreaker`, `AdapterCircuitBreakerProvider`, `AdapterCircuitState`, `AdapterCircuitOutcome`, `AdapterCircuitAdmission`, `AdapterCircuitBreakerEvent` and `AdapterCircuitBreakerManualOverride`. `AdapterCircuitAdmission.generation` identifies the epoch an admission was granted in.
|
|
58
|
+
- `AdapterCircuitBreakerConfig` gains the optional `manualOverride` (`"force_open"` or `"force_closed"`) and `probeLeaseMs` fields. No existing field changed.
|
|
59
|
+
- The breaker is per process: state is in memory, is not shared between replicas and is lost on restart.
|
|
60
|
+
- It goes closed → open → half-open (one probe) → closed, over a rolling `windowMs` and a `cooldownMs`. At most one half-open probe is outstanding until it records or its lease expires. The lease is the new optional `probeLeaseMs` (default `max(cooldownMs, windowMs)`), independent of the cooldown. `check()` is a side-effect-free preview.
|
|
61
|
+
- Only outcomes from the current epoch change state. The breaker's epoch advances on every state change, manual-override change and probe grant, and each admission carries it as `generation`. An outcome admitted in an earlier epoch is dropped: it never closes or reopens the breaker, never resolves or frees the probe slot, and is not counted toward a later trip. For example, a replaced probe that fails after its replacement closed the breaker does not reopen it. A direct `record()` call without an admission, or with one that has no `generation`, is treated as current.
|
|
62
|
+
- `failureThreshold` is a failure count, matching the .NET port. Once `minimumSamples` (default `failureThreshold`) outcomes are in the window, the breaker opens on `failureThreshold` consecutive failures, or on a failure ratio of at least `failureThreshold / minimumSamples`.
|
|
63
|
+
- Outcomes `transient_failure`, `timeout`, `ambiguous` and unclassified thrown errors count as failures. `permanent_failure` is ignored.
|
|
64
|
+
- **`executeWithAdapterRetry`** takes an optional `circuitBreaker`. Without it, behaviour is unchanged.
|
|
65
|
+
- With it, admission is checked immediately before every attempt, with no await between admission and `execute`, and the outcome of every attempt that reached `execute` is recorded.
|
|
66
|
+
- New optional `prepareAttempt(attempt, maxAttempts)` hook, awaited before admission. If it throws, no admission is taken and no outcome is recorded; the error is classified and retried or rethrown like an error thrown by `execute`.
|
|
67
|
+
- In `enforce` mode, a refused attempt is not executed and `AdapterCircuitOpenError` is thrown (`code: "ADAPTER_CIRCUIT_OPEN"`, with `adapterKey`, `attempt` and `retryAfterMs` when known).
|
|
68
|
+
- When the breaker opens between attempts, retrying stops and the same error is thrown.
|
|
69
|
+
- `monitor` mode never blocks. It reports `would_reject` telemetry through `onEvent`, and the callback's errors are swallowed. Events are delivered after the breaker operation that produced them commits, never during a decision or a record, so a callback that calls back into the breaker (for example to force it open) sees committed state, and its change applies from the next decision on. A callback never revokes an admission that is already committed; the README lists the two edge cases this leaves (a breaker call made from inside a callback, and `cooldownMs: 0`).
|
|
70
|
+
- **`@fabricorg/platform-host`.**
|
|
71
|
+
- Each Host instance keeps one breaker per `adapterType:operation`. A breaker is created on the adapter's first call, and only for adapters that declare `circuitBreaker`.
|
|
72
|
+
- Host now writes each attempt number to the adapter invocation record in `prepareAttempt`, before breaker admission, instead of inside `execute`. A store failure there is classified and retried as before, and is not counted against the breaker.
|
|
73
|
+
- The new `adapterCircuitBreakers` option supplies a shared `AdapterCircuitBreakerRegistry` or a custom provider (shared breakers, telemetry, manual override).
|
|
74
|
+
- A circuit-open refusal means the adapter never ran. Host marks the adapter invocation `failed` and appends `AdapterInvocationFailed` with the additive payload fields `reason: "circuit_open"` and `retryAfterMs`. The invocation then fails with status `failed`, the same path as an unsuccessful adapter result.
|
|
75
|
+
- A refusal is never routed to `reconciliation_required`, and it is never classified as a timeout. Domain events already appended in the `before_adapters` phase are kept.
|
|
76
|
+
- `createGovernedActionHost()` now throws at construction if an adapter declares an invalid `circuitBreaker` config, for example a `failureThreshold` below 1. Such a config was previously ignored.
|
|
77
|
+
- **Peer range:** Host now requires `@fabricorg/platform` `^1.8.0` (was `^1.2.0`), because it imports the new breaker exports at runtime. The manifest declares it as `workspace:^`, like the other workspace packages that peer on Platform, so the published range is the Platform version released with Host. A literal `^1.8.0` would fail the packed-consumer check (`pnpm verify:packed`) until the version PR publishes 1.8.0. A side effect: each later Host release will require at least the Platform version that is current when that Host is released.
|
|
78
|
+
|
|
79
|
+
### Patch Changes
|
|
80
|
+
|
|
81
|
+
- e0f7ca7: Host now imports `canonicalJson` from `@fabricorg/platform` instead of defining its own copy (issue #24). The function moved unchanged, so parameter digests and the external-completion digest are byte-identical, and Host still exports `canonicalJson`, `digestParameters` and `PARAMETER_DIGEST_ALGORITHM`. Host's peer range on `@fabricorg/platform` stays `workspace:^`, so the published range is the Platform version released alongside this Host release, which includes `canonicalJson`.
|
|
82
|
+
|
|
3
83
|
## 7.3.0
|
|
4
84
|
|
|
5
85
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -47,6 +47,39 @@ reclaimed so its token advances. An expired worker cannot finalize after another
|
|
|
47
47
|
`idempotencyKey` to `submitAction()` so retries resolve
|
|
48
48
|
to the original tenant-scoped invocation instead of creating another mutation.
|
|
49
49
|
|
|
50
|
+
For transient governance dependency failures, throw `RetryableExecutionError` from
|
|
51
|
+
`@fabricorg/platform` (also exported from `@fabricorg/platform/policies`). The retry region covers
|
|
52
|
+
parameter validation, mutation-governance resolution, HITL evaluation, entitlement and execution
|
|
53
|
+
authorization (including the `authorize` fallback), policy resolution, and policy evaluation. It ends
|
|
54
|
+
immediately before the first policy-evaluation record is written. Host rethrows the original error
|
|
55
|
+
without a failure/status write when:
|
|
56
|
+
|
|
57
|
+
- the invocation has a lease owner and a defined lease token;
|
|
58
|
+
- the action declares `idempotent: true`;
|
|
59
|
+
- `attemptCount` is below `retryableExecutionMaxAttempts`, a positive integer option defaulting to 5; and
|
|
60
|
+
- execution is not a `resumeApprovedInvocation()` call.
|
|
61
|
+
|
|
62
|
+
The worker reports the original error through `onError` and stops its heartbeat. Its cycle result
|
|
63
|
+
and `worker.cycle.completed` telemetry include `deferred` only when the count is greater than zero;
|
|
64
|
+
deferrals do not increment `failed`. The worker counts by error type: a `RetryableExecutionError`
|
|
65
|
+
that a store wrapper throws from Host's own reads or status writes also lands in `deferred`. In that
|
|
66
|
+
case the row likewise stays leased for a later claim, but outside this cap. Retry becomes eligible when the current lease expires, so the
|
|
67
|
+
delay is the remaining lease duration, followed by the worker's next claim; there is no immediate
|
|
68
|
+
retry or separate backoff. The next claim increments `attemptCount`. At the cap, or for an unleased,
|
|
69
|
+
non-idempotent, or approval-resume invocation, Host records `failed` with the reason retry was refused,
|
|
70
|
+
through the error-text redactor. Approval resume is excluded because its temporary lease does not
|
|
71
|
+
establish that a recovery worker is configured. Ordinary thrown errors retain terminal failure behavior.
|
|
72
|
+
|
|
73
|
+
The retry region includes Host's single-record store writes for mutation-governance and HITL
|
|
74
|
+
checkpoints. Later attempts reuse these persisted decisions through the `mutationFootprint` and
|
|
75
|
+
`hitlRoute` guards. Callbacks should be safe to repeat and return stable invocation-scoped evidence.
|
|
76
|
+
A deferral happens before the attempt writes any policy evidence: the complete definition batch must
|
|
77
|
+
evaluate before Host starts appending policy records. Policy persistence, the `ComplianceBlocked`
|
|
78
|
+
event, obligation listing/resolution/persistence, state-machine checks, the handler transaction,
|
|
79
|
+
handlers, adapters, and finalization retain their existing failure/reconciliation/recovery semantics.
|
|
80
|
+
A retryable obligation-resolver error fails terminally with the policy evidence from that attempt.
|
|
81
|
+
Preview evaluation does not enter this execution catch.
|
|
82
|
+
|
|
50
83
|
New rows bind that key to the action version, actor/authority binding, and a versioned canonical
|
|
51
84
|
parameter digest. Host 3 defaults to `enforce`, which throws `IdempotencyConflictError`; configure
|
|
52
85
|
`idempotency.conflictMode` as `audit-only` for a preflight soak. Legacy null-digest rows skip only the
|
|
@@ -65,6 +98,52 @@ may or may not have been applied — is routed to `reconciliation_required` with
|
|
|
65
98
|
`ExternalReconciliation` evidence and an `AdapterInvocationAmbiguous` lifecycle event rather than
|
|
66
99
|
blindly retrying an unsafe effect.
|
|
67
100
|
|
|
101
|
+
### Adapter circuit breakers
|
|
102
|
+
|
|
103
|
+
An adapter that declares `circuitBreaker` gets a breaker from Host. By default each Host instance
|
|
104
|
+
keeps its own `AdapterCircuitBreakerRegistry`: one in-process breaker per `adapterType:operation`,
|
|
105
|
+
created on the adapter's first call from that adapter's own config. Adapters without a config get
|
|
106
|
+
none, and behave as before. `createGovernedActionHost()` throws if a declared config is invalid.
|
|
107
|
+
Breaker state lives in the Host's process memory: it is not shared between replicas or workers and
|
|
108
|
+
is lost on restart. Pass `adapterCircuitBreakers` to share a registry between Host instances in one
|
|
109
|
+
process, to receive breaker telemetry (`new AdapterCircuitBreakerRegistry({ onEvent })`), to set a
|
|
110
|
+
manual override (`registry.get(type, operation)?.setManualOverride("force_open")`), or to supply a
|
|
111
|
+
custom `AdapterCircuitBreakerProvider`.
|
|
112
|
+
|
|
113
|
+
The default key is shared across tenants, spaces, and vendors within one registry: one tenant's
|
|
114
|
+
failures can open the breaker for another. For tenant/space/vendor isolation, inject a scoped registry:
|
|
115
|
+
|
|
116
|
+
```ts
|
|
117
|
+
import { AdapterCircuitBreakerRegistry, scopedAdapterCircuitBreakerKey } from "@fabricorg/platform/adapters";
|
|
118
|
+
|
|
119
|
+
const adapterCircuitBreakers = new AdapterCircuitBreakerRegistry({ key: scopedAdapterCircuitBreakerKey });
|
|
120
|
+
// Pass adapterCircuitBreakers to createGovernedActionHost({ store, registry, adapters, ... }).
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Host passes the trusted `{ tenantId, spaceId }` invocation context to `forAdapter(adapter, scope)`;
|
|
124
|
+
providers receive it from `@fabricorg/platform-host` 7.5.0 on. Existing one-argument providers remain compatible. The first config
|
|
125
|
+
seen for a registry key wins; later conflicting configs reuse the existing breaker without throwing.
|
|
126
|
+
Adapters sharing a key should use the same configuration. The helper URI-encodes components and
|
|
127
|
+
preserves a readable `adapterType:operation` suffix for ordinary identifiers. Missing space or vendor
|
|
128
|
+
is distinct from an empty string; `vendor` is required by the TypeScript adapter contract.
|
|
129
|
+
Calls made outside Host without scope use a separate vendor-specific `unscoped` namespace and share
|
|
130
|
+
breakers there; tenant isolation requires scope. Use non-secret identifiers because keys appear in
|
|
131
|
+
telemetry and errors. Breakers are never evicted: a scoped key creates one breaker per tenant, space, vendor and operation
|
|
132
|
+
for the life of the registry, and those keys are high-cardinality, so do not use them as metric labels.
|
|
133
|
+
For a custom-keyed breaker, use
|
|
134
|
+
`registry.getByKey(scopedAdapterCircuitBreakerKey(adapter, scope))`; `get(type, operation)` continues
|
|
135
|
+
to look up the default key.
|
|
136
|
+
|
|
137
|
+
The breaker is consulted immediately before every attempt, including retries. Host writes the attempt number to the adapter invocation record first, in `prepareAttempt`, so nothing awaits between admission and the adapter call; a store failure there is not counted against the breaker. In `monitor` mode it
|
|
138
|
+
never blocks. In `enforce` mode, while it is open, Host does not call the adapter: the adapter
|
|
139
|
+
invocation is marked `failed`, `AdapterInvocationFailed` is appended with
|
|
140
|
+
`payload.reason: "circuit_open"` (and `retryAfterMs` when known), and the invocation fails with
|
|
141
|
+
status `failed` exactly like an unsuccessful adapter result. Because the adapter never ran, a
|
|
142
|
+
circuit-open refusal is never routed to `reconciliation_required`. Domain events already appended
|
|
143
|
+
in the `before_adapters` phase stay appended. If the breaker opens between retry attempts, Host stops
|
|
144
|
+
retrying and records the refusal. See the `@fabricorg/platform` README for thresholds and outcome
|
|
145
|
+
mapping.
|
|
146
|
+
|
|
68
147
|
## Execution authorization
|
|
69
148
|
|
|
70
149
|
Submission authorization proves that a caller may create an invocation. Applications that delegate
|
|
@@ -136,12 +215,18 @@ must all use the same database transaction.
|
|
|
136
215
|
`PostgresPlatformHostTransactionProvider`. The application owns that narrow binder because only the
|
|
137
216
|
application knows how its `TDb` is rebound to the transaction's SQL client.
|
|
138
217
|
|
|
139
|
-
|
|
140
|
-
|
|
218
|
+
With `transactionWithEvents`:
|
|
219
|
+
|
|
220
|
+
- `before_adapters`: handler domain writes, declared domain events, and the Host-generated
|
|
221
|
+
`StateTransitioned` fact commit or roll back together.
|
|
222
|
+
- `after_adapters`: declared domain events, the Host-generated `StateTransitioned` fact, and
|
|
223
|
+
invocation finalization commit together after required adapter steps succeed or accept a
|
|
224
|
+
parked external handoff. Handler writes have already committed separately. A finalization
|
|
141
225
|
failure leaves the invocation recoverable; succeeded adapter checkpoints are not repeated.
|
|
142
226
|
|
|
143
|
-
Stores without
|
|
144
|
-
|
|
227
|
+
Stores without `transactionWithEvents` retain the legacy non-atomic boundary for domain writes,
|
|
228
|
+
events, and invocation persistence. A partial append or finalization failure can leave some
|
|
229
|
+
records persisted; this path does not provide atomic domain-write/event persistence.
|
|
145
230
|
|
|
146
231
|
`createDurableSagaParentLifecycle()` requires this capability. Each parent lifecycle transition uses a
|
|
147
232
|
stable `lifecycleId` as its causation identity, appends its audit event, and updates the parent invocation
|
|
@@ -207,6 +292,59 @@ and pass an opaque identifier instead; action parameters are intentionally durab
|
|
|
207
292
|
Hosts should additionally configure `redactActionParameters` as a fail-safe allowlist for actions
|
|
208
293
|
whose ingress is adjacent to secret material; redaction runs before the invocation row is created.
|
|
209
294
|
|
|
295
|
+
Three optional `PlatformHostOptions` hooks cover other Host-recorded values.
|
|
296
|
+
|
|
297
|
+
| Hook | When it runs and what uses its return value |
|
|
298
|
+
| --- | --- |
|
|
299
|
+
| `redactAdapterOutput(actionId, adapterType, operation, output)` | Once after a successful adapter result is converted to output, before handoff handling. The adapter row, `AdapterInvocationSucceeded` payload, and `extractExecutionAttestation` input use the redacted output. |
|
|
300
|
+
| `redactErrorText(context, message)` | At the origin of caught handler/execution errors, adapter result errors and ambiguous-outcome reasons, and completion errors. The invocation/adapter errors, reconciliation messages and reasons, completion evidence, failure events, and returned error reuse that text. |
|
|
301
|
+
| `redactExternalCompletionResult(actionId, provider, result)` | After a completion matches its invocation, before merging, result-schema validation, persistence, and digesting. See [External completion](#external-completion). |
|
|
302
|
+
|
|
303
|
+
`ErrorTextRedactionContext` is exported. It contains `actionId` and `source` (`handler`, `adapter`,
|
|
304
|
+
`external_completion`, or `execution`); adapter errors also supply `adapterType`, `operation`, and
|
|
305
|
+
`provider` (the adapter vendor), while completion errors supply `provider`. Exceptions from
|
|
306
|
+
application execution hooks such as `getInput` and `extractExecutionAttestation` use `execution`.
|
|
307
|
+
Host-authored fixed messages and structured identifiers, including `externalReference`, bypass this
|
|
308
|
+
hook. Circuit-open errors and consistency errors use adapter context because an adapter can supply
|
|
309
|
+
those messages or their interpolated values. When Host adds context around a caught store error,
|
|
310
|
+
only the caught part is redacted. Empty originating error text bypasses the hook unchanged. For non-empty input, a hook that throws,
|
|
311
|
+
returns a non-string, or returns an empty string uses `Error text withheld: the error-text redactor failed.` on execution paths. These paths retain
|
|
312
|
+
their failure classification when the error can be persisted. For a completion error, the same hook
|
|
313
|
+
failures refuse with non-retryable `redaction_failed` before writes, allowing a sender to submit the
|
|
314
|
+
callback after the redactor is repaired.
|
|
315
|
+
|
|
316
|
+
If `redactAdapterOutput` throws or returns null, an array, or a non-object value, the external effect
|
|
317
|
+
has already happened. Host persists the `reconciliation_required` verdict before the adapter row.
|
|
318
|
+
It retries an idempotent verdict write once after a store error; a lost lease is not retried. After
|
|
319
|
+
the verdict is durable, Host emits the retained handoff's `ExternalOperationAccepted`, emits
|
|
320
|
+
`AdapterInvocationSucceeded` with redacted input and `outputWithheld: true` without output, appends
|
|
321
|
+
fixed-text reconciliation evidence when the store supports it, and marks the adapter `succeeded`.
|
|
322
|
+
The events use the normal step deduplication keys. A first accepted reference is retained as
|
|
323
|
+
`pendingCompletion`, so its callback can still land as evidence. References come from the original
|
|
324
|
+
adapter result. Refused handoffs also persist their verdict before evidence and the adapter marker.
|
|
325
|
+
|
|
326
|
+
Once the reconciliation verdict is durable, direct execution returns it and recovery excludes it.
|
|
327
|
+
A later evidence or adapter-marker write failure does not change it to completed or failed; Host
|
|
328
|
+
throws `RecoverableFinalizationError`. The verdict, evidence and marker are separate writes, so an
|
|
329
|
+
evidence-write failure can leave a partial audit trail. If both verdict-write attempts fail, the
|
|
330
|
+
adapter can run again on recovery, the same exposure as a crash mid-step. Adapters behind these paths
|
|
331
|
+
should be idempotent or carry a stable idempotency key. A lost lease can also prevent this branch
|
|
332
|
+
from establishing its own durable verdict; these are not atomic guarantees.
|
|
333
|
+
|
|
334
|
+
`extractExecutionAttestation` receives the redacted output, so an extractor that needs a stripped
|
|
335
|
+
field can fail after the external effect, following the existing extractor-failure behavior.
|
|
336
|
+
|
|
337
|
+
`redactActionParameters` and `redactAdapterInput` behavior is unchanged. Independently of hook
|
|
338
|
+
configuration, a stale worker that loses its lease inside an adapter step rethrows without marking
|
|
339
|
+
the adapter failed, preserving the new owner's row. Refused-handoff ordering and audit events also
|
|
340
|
+
change without the new hooks.
|
|
341
|
+
|
|
342
|
+
Applications own the safety of domain events/results, policy and HITL evidence, explicit
|
|
343
|
+
attestations/reconciliations, saga lifecycle inputs, approval reasons, and outbox publisher errors.
|
|
344
|
+
Parameter-schema issue messages used for `validation_failed`, the `HITL rejected` reason, policy
|
|
345
|
+
block reasons, and action-result validation paths are not passed through `redactErrorText`.
|
|
346
|
+
These hooks do not rewrite historical records or exceptions delivered to a worker's `onError`.
|
|
347
|
+
|
|
210
348
|
## Submission snapshot
|
|
211
349
|
|
|
212
350
|
`submitAction` reads the scalar fields of the caller's `SubmitActionInput` once, before its first
|
|
@@ -371,13 +509,87 @@ controlled deployment migration step, and hydrate projections from `listEvents()
|
|
|
371
509
|
uses ordered immutable checksums, a migration ledger, and a transaction-scoped advisory lock. It
|
|
372
510
|
remains safe to call at startup, but deployment-owned migrations are preferred.
|
|
373
511
|
|
|
374
|
-
State-machine transition checks run inside the Host mutation unit of work.
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
512
|
+
State-machine transition checks run inside the Host mutation unit of work, before the handler. Stores implementing `transactionWithEvents`
|
|
513
|
+
should expose transaction-scoped `getEntityState()` so the guard reads the same database snapshot
|
|
514
|
+
used by the handler. Existing custom stores without that optional transaction method fall back
|
|
515
|
+
to the store-level authoritative read.
|
|
516
|
+
|
|
517
|
+
An idempotent action may resume in its target state despite a missing self-loop only when the
|
|
518
|
+
event that set the current recorded state belongs to that same `actionInvocationId`. The handler
|
|
519
|
+
can rerun while stable event IDs suppress duplicate events. A new invocation validates normally;
|
|
520
|
+
model a self-loop if it should re-apply the state, or use the same idempotency key to deduplicate
|
|
521
|
+
to the original invocation at submission.
|
|
522
|
+
|
|
523
|
+
The optional `getEntityStateSource(tenantId, spaceId, entityType, entityId)` store and transaction
|
|
524
|
+
method returns `{ state, actionInvocationId }` from the same event as `getEntityState()`, or
|
|
525
|
+
`undefined` when provenance is unavailable. Memory transaction reads include pending events.
|
|
526
|
+
Host prefers the transaction method, then the store method. If neither is implemented, it uses
|
|
527
|
+
transaction `listEvents()`, otherwise store `listEvents()` when available, selecting the subject's
|
|
528
|
+
highest-sequence event with a string `payload.toState`. The source state and invocation writer
|
|
529
|
+
must match the current state and invocation. Missing provenance leaves normal validation in force;
|
|
530
|
+
custom stores can continue compiling without the optional method.
|
|
531
|
+
|
|
532
|
+
After a valid transition for a binding with a non-empty entity ID and target, Host appends
|
|
533
|
+
`StateTransitioned` immediately after the handler's domain events, following the action's
|
|
534
|
+
`eventPhase`. Its subject is the bound entity and its version-1 payload is
|
|
535
|
+
`{ entityType, entityId, fromState, toState, causedByAction }`. `fromState` is the trusted state
|
|
536
|
+
read before the handler, and `causedByAction` is the action ID. The event uses a stable
|
|
537
|
+
invocation-scoped ID and is publishable through the outbox by default; `outbox.shouldPublish`
|
|
538
|
+
can filter it. It does not need an `emitsEvents` declaration from the action.
|
|
539
|
+
|
|
540
|
+
For `before_adapters`, the transition is appended with the handler's domain events. For
|
|
541
|
+
`after_adapters`, it is deferred to finalization after required adapter steps succeed or
|
|
542
|
+
accept a parked external handoff, immediately after domain events. Adapter failure withholds
|
|
543
|
+
both domain events and the transition, leaving Host's recorded entity state unchanged by that
|
|
544
|
+
invocation even though handler writes may have committed. With `transactionWithEvents`, the
|
|
545
|
+
transition shares the transaction used for domain events in the selected phase; otherwise it
|
|
546
|
+
follows the same legacy non-atomic boundary as domain events.
|
|
547
|
+
|
|
548
|
+
Invalid transitions, failed handlers, replay-only acceptance in the target state, and bindings
|
|
549
|
+
without a non-empty entity ID or target do not produce a Host transition fact. If the handler
|
|
550
|
+
returns its own `StateTransitioned` with the bound entity's subject type and ID, Host suppresses
|
|
551
|
+
its generated fact in either event phase. Applications using that workaround should remove it
|
|
552
|
+
to use Host emission.
|
|
553
|
+
|
|
554
|
+
Host's new transition fact advances recorded state even if handler domain events omit `toState`.
|
|
555
|
+
Later calls validate against that advanced state; a new invocation can fail if the
|
|
556
|
+
machine lacks the corresponding edge or self-loop. Model those edges, or submit with the same
|
|
557
|
+
idempotency key to deduplicate to the original invocation. Applications already recording `toState` in
|
|
558
|
+
domain events retain their existing state advancement behavior when that `toState` matches the
|
|
559
|
+
binding's target; otherwise Host's later fact sets the recorded state to the target.
|
|
378
560
|
|
|
379
561
|
## External completion
|
|
380
562
|
|
|
563
|
+
Configure `redactExternalCompletionResult` to withhold sensitive completion fields. Host applies it
|
|
564
|
+
once to a supplied result (including a result attached to a failed callback). The redacted value
|
|
565
|
+
feeds `externalCompletion.result`; for a completed outcome it also feeds the adapter output,
|
|
566
|
+
`ExternalOperationCompleted` payload, merged invocation result, and returned result. A supplied
|
|
567
|
+
`completion.error` passes through `redactErrorText` before recording, including when attached to a
|
|
568
|
+
completed outcome. A missing error uses Host's fixed failure message, which bypasses the hook.
|
|
569
|
+
|
|
570
|
+
The recorded digest hashes the redacted result and error when hooks are configured. A repeat first compares the raw incoming digest in memory before invoking any hook, recognizing
|
|
571
|
+
legacy records even if a newly configured redactor would fail. Otherwise it compares the redacted digest. Newly stored digests use the redacted representation.
|
|
572
|
+
Contradiction detection therefore compares what redactors retain; differences they discard are
|
|
573
|
+
considered the same news. The compatibility match leaves historical records intact.
|
|
574
|
+
|
|
575
|
+
Redactors should be deterministic and stable across deploys. A changed redactor can make a repeat
|
|
576
|
+
look like different evidence and trigger contradiction handling.
|
|
577
|
+
|
|
578
|
+
The object hooks accept non-null, non-array objects, including class instances and Dates, matching
|
|
579
|
+
the existing adapter-output `isRecord` predicate. Strings, arrays and null are rejected.
|
|
580
|
+
|
|
581
|
+
If completion-result redaction throws or returns null, an array, or a non-object, or completion
|
|
582
|
+
error-text redaction throws or returns a non-string or empty string for non-empty input, `ExternalCompletionError.refusal` is `redaction_failed` and
|
|
583
|
+
`retryable` is false. Host refuses it before writing the invocation, adapter row, event, or
|
|
584
|
+
reconciliation evidence; fix the redactor before submitting again. Consumers with exhaustive
|
|
585
|
+
`ExternalCompletionRefusal` switches need a `redaction_failed` case. Other permanent refusals are
|
|
586
|
+
`not_found`, `not_awaiting`, `immediate_contract`, `reference_mismatch`, `provider_mismatch`,
|
|
587
|
+
`invalid_completion`, and `result_invalid`; `still_executing` is retryable.
|
|
588
|
+
|
|
589
|
+
While a redaction hook is failing, a contradicting repeat that needs redaction is also refused with
|
|
590
|
+
`redaction_failed`, without recording a contradiction. A matching legacy raw digest still returns
|
|
591
|
+
before hooks run. Empty completion error input bypasses the error-text hook unchanged.
|
|
592
|
+
|
|
381
593
|
An action declaring `completion: "accepted"` or `"long-running"` may hand its work to an external
|
|
382
594
|
system. The adapter returns `acceptedExternalOperation: { externalReference }`; the Host parks the
|
|
383
595
|
invocation as `running` with no lease, emits `ExternalOperationAccepted`, and leaves it for the
|
|
@@ -399,7 +611,8 @@ reference, outcome, result, error, and evidence. A contradictory one, a flipped
|
|
|
399
611
|
outcome with different evidence, moves the invocation to
|
|
400
612
|
`reconciliation_required`, keeps the first outcome on record, logs the second, and emits
|
|
401
613
|
`ExternalOperationContradicted`. Under an atomic store the read and write happen under one lock. The
|
|
402
|
-
callback's result is
|
|
614
|
+
callback's redacted result is merged with the invocation result, stripped of private fields, and
|
|
615
|
+
validated against the action's result schema before writes. The stored merge is what is validated.
|
|
403
616
|
An invocation that ended for another reason while a handoff was pending keeps its status and records
|
|
404
617
|
the callback as evidence. Declare `completionDeadlineMs` on `long-running` actions;
|
|
405
618
|
`reconcileOverdueCompletions()` moves overdue handoffs to reconciliation while keeping the handoff so
|
|
@@ -431,12 +644,20 @@ cast in an index expression is not immutable and PostgreSQL refuses it.
|
|
|
431
644
|
|
|
432
645
|
## Tenant isolation
|
|
433
646
|
|
|
434
|
-
`tenantIsolationChecks()` and `runTenantIsolationChecks(subject)`
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
647
|
+
`tenantIsolationChecks()` and `runTenantIsolationChecks(subject)` run eight checks against a host
|
|
648
|
+
and its store. They prove scope on the invocation reads and writes, idempotency keys, event streams,
|
|
649
|
+
worker claims, external completion, an optional governed read, and adapter checkpoint reads and
|
|
650
|
+
writes they probe. The subject supplies a host, a recoverable store, a registered action, and two
|
|
651
|
+
scopes in different tenants. Adapter checks create probes in dedicated spaces and retire them afterwards.
|
|
652
|
+
|
|
653
|
+
Host passes `AdapterInvocationScope` from the store-loaded action invocation to
|
|
654
|
+
`getAdapterInvocation(id, scope)` and `updateAdapterInvocation(id, patch, scope)`. Custom stores
|
|
655
|
+
must match `tenantId`, `spaceId`, and `actionInvocationId` when scope is supplied. A mismatch returns
|
|
656
|
+
`undefined` on read and leaves the row unchanged on update (a not-found error is permitted).
|
|
657
|
+
The trailing argument is optional to preserve existing custom-store and direct-call compatibility;
|
|
658
|
+
calls without it retain ID-only behavior in the shipped stores. The two adapter isolation checks
|
|
659
|
+
fail for stores that ignore scope. Deployments running `runTenantIsolationChecks` against a custom
|
|
660
|
+
store will see new failures until that store honours the argument.
|
|
440
661
|
|
|
441
662
|
## Runtime observability and health
|
|
442
663
|
|