@fabricorg/platform-host 7.4.0 → 7.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +69 -0
- package/README.md +295 -15
- package/dist/index.cjs +540 -73
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +150 -17
- package/dist/index.d.ts +150 -17
- package/dist/index.js +541 -74
- package/dist/index.js.map +1 -1
- package/package.json +4 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,74 @@
|
|
|
1
1
|
# @fabricorg/platform-host
|
|
2
2
|
|
|
3
|
+
## 7.6.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 74295fd: Add optional `updateAdapterInvocation` and `appendExternalReconciliation` members to `PlatformHostMutationTransaction` (issue #71). The adapter update retains the store method's tenant/space/invocation scope semantics. Memory transactions buffer these writes and undo ledger writes if commit fails; PostgreSQL delegates them to the transaction-bound scoped store.
|
|
8
|
+
|
|
9
|
+
Host uses these members for withheld-output and refused-handoff finalization. Evidence events and the adapter's succeeded marker precede the fenced terminal verdict inside the transaction. Withheld-output governance reconciliation shares that transaction when applicable. Leased atomic finalization also requires transaction-scoped `updateLeasedActionInvocation`; missing required members select the verdict-first fallback before evidence writes. Host retries the whole transaction once on a non-lease error, then selects the verdict-first fallback if the retry also fails with a non-lease error. Lease loss on either attempt raises `RecoverableFinalizationError` without fallback. Status telemetry fires after an atomic commit or through fallback verdict persistence. Refused-handoff completion reconciliation logging and its tolerated-failure error update remain after commit. Event IDs, payloads, returned results, and pending-completion selection are unchanged.
|
|
10
|
+
|
|
11
|
+
**Release note: failure semantics (issue #71).** When atomic finalization succeeds on a store with the required transaction members, the degraded verdict and its evidence commit together. A rolled-back atomic attempt leaves the invocation and adapter row `running` until the retry or fallback commits. Repeated non-lease transaction failures select the verdict-first fallback so the invocation can reach terminal `reconciliation_required` even when atomic evidence cannot be written. If the fallback evidence write also fails, Host throws `RecoverableFinalizationError` after the verdict commits, retaining the prior terminal-verdict/evidence-gap behavior. Lease loss or failure to persist the fallback verdict can still leave the invocation reclaimable; the adapter then re-runs under its `adapterInvocationId` idempotency key, as after a crash immediately following the adapter return. Previously a terminal verdict could remain with missing evidence and a running adapter row. Non-atomic stores and transactions lacking the required members keep the old verdict-first order and its evidence gap. Custom stores must implement the new transaction members in the same transaction to get the atomic behavior. Finalization failures retain `RecoverableFinalizationError`; adapters still need idempotency or a stable idempotency key.
|
|
12
|
+
|
|
13
|
+
A store wrapper that spreads a built-in transaction object (`...transaction`) inherits the new members: Host commits these writes through the inner transaction rather than through the wrapper's store-level overrides of `updateAdapterInvocation` or `appendExternalReconciliation`.
|
|
14
|
+
|
|
15
|
+
A successful COMMIT with a lost acknowledgement can cause the leased retry to report lease loss (`RecoverableFinalizationError`) even though the stored state is complete, as with the previous verdict retry.
|
|
16
|
+
|
|
17
|
+
- 5d93521: Add `ReconciliationRequiredError` and `isReconciliationRequiredError` to Platform's root and policies exports (issue #70). The error accepts a message and optional cause; its stable name and `EXECUTION_RECONCILIATION_REQUIRED` code support recognition across package copies.
|
|
18
|
+
|
|
19
|
+
Host honours this signal at the handler-transaction catch, including a store wrapper detecting that its own recovery receipt no longer matches the action's declared events or result schema before handing it back. Wrappers cannot tell which Host transaction they wrap: the same API also serves finalization and external completion. Translate only conditions the store detects itself, not Host internal errors matched by message text; those errors have no public stable code or message contract. Put non-sensitive operator detail in the message because `cause` is not stored. It redacts the message, persists terminal `reconciliation_required`, and returns consistency and HITL metadata. The existing status telemetry fires through persistence. Re-execution returns the terminal record without running the handler; adapters do not run after this signal. Atomic `transactionWithEvents` stores roll back the attempt's domain writes, domain events, and `StateTransitioned` together. On the `transaction`-only fallback, Host appends events directly and the built-in stores do not roll back: domain writes and any events already appended during the attempt stay persisted beside the `reconciliation_required` record, as they already do beside `failed`. A custom fallback transaction may roll back domain writes, but Host does not make event appends atomic with those writes.
|
|
20
|
+
|
|
21
|
+
**Release note: failure semantics (issue #70).** Previously these handler-transaction throws recorded terminal `failed`. Opting into `ReconciliationRequiredError` now requests an operator decision where a domain effect may already have committed. Ordinary handler errors, pre-handler governance/validation, state guards, adapters, and finalization retain their existing failure or recovery semantics. `after_adapters` finalization still uses `RecoverableFinalizationError`. Workers continue counting returned reconciliation results as `failed`. No new event is introduced. Host's Platform peer remains `workspace:^` so published packages require the Platform release containing the runtime exports.
|
|
22
|
+
|
|
23
|
+
- 73d5f98: Preserve the validated transition for idempotent `after_adapters` actions in the optional durable invocation field `pendingStateTransition` (issue #69). Host saves the entity type, entity ID, original state, and target. With a mutation transaction, it writes through transaction-scoped `updateActionInvocation` after the handler succeeds and result/events validate, just before the callback returns. This avoids holding the invocation row lock during the handler while keeping the evidence atomic with domain writes. Without a mutation transaction, Host writes before the handler. Non-idempotent actions do not write the field. The saved transition is retained as evidence after completion. Memory and PostgreSQL stores persist it through invocation patches; PostgreSQL supports both ordinary and leased updates.
|
|
24
|
+
|
|
25
|
+
**Release note: recovery semantics (issue #69).** A recovered replay admitted by the same-invocation state shortcut can emit the original `StateTransitioned` after adapters when the saved entity type, entity ID, and target match the current binding and state. It uses the saved `fromState`, the existing payload shape, and the stable `state:transition` event ID; built-in event and outbox stores deduplicate that ID. A handler-supplied transition for the same subject still suppresses Host emission. Custom stores must persist and reload `pendingStateTransition`, including through transaction-scoped `updateActionInvocation`, to get this behavior. Invocations started before upgrading have no saved transition and get no re-emitted event. Missing or mismatched evidence preserves replay acceptance without fabricating a prior state. PostgreSQL `ensureSchema()` adds the `pending_state_transition jsonb` column through additive migration 7, preserving the checksums of earlier migrations. `before_adapters` continues to record the transition with the handler's events and does not write this field.
|
|
26
|
+
|
|
27
|
+
## 7.5.0
|
|
28
|
+
|
|
29
|
+
### Minor Changes
|
|
30
|
+
|
|
31
|
+
- e32ee98: Add opt-in tenant/space/vendor circuit-breaker isolation (issue #52). Platform exports `AdapterCircuitBreakerScope` and `scopedAdapterCircuitBreakerKey` from the root and adapters subpath, accepts optional scope in providers, and adds registry `key` and `getByKey` APIs. Host passes trusted invocation tenant/space scope to providers.
|
|
32
|
+
|
|
33
|
+
The default `adapterType:operation` key and its shared failure domain are unchanged. Consumers can inject `new AdapterCircuitBreakerRegistry({ key: scopedAdapterCircuitBreakerKey })` to prevent one tenant's failures from refusing another tenant's calls. The helper encodes components, distinguishes absent space/vendor from empty values, and uses a separate vendor-specific unscoped namespace when scope is absent. Calls without scope share that namespace; tenant isolation requires scope. Keys appear in telemetry and errors, so use non-secret identifiers.
|
|
34
|
+
|
|
35
|
+
The first config seen for a key wins, including custom keys; later conflicting configs reuse the existing breaker without throwing. Existing one-argument providers remain compatible. Host's `workspace:^` Platform peer declaration is unchanged: passing an extra argument is compatible with older implementations, and Host adds no runtime import of the new helper. Release tooling resolves the workspace peer against the released Platform version.
|
|
36
|
+
|
|
37
|
+
- cfada18: Restrict the idempotent same-state replay shortcut to the invocation that wrote the current recorded state (issue #55).
|
|
38
|
+
|
|
39
|
+
**Release note: validation change.** The shortcut now applies only to a retry of the invocation whose event set the current recorded state. A new invocation re-applying an already-reached state is validated normally and fails unless the state machine defines that edge. Model a self-loop for a new mutation, or submit with the same idempotency key to deduplicate to the original invocation at submission. An idempotent action targeting the machine's initial state on an entity with no recorded state (or a binding with no entity ID), previously admitted by the shortcut, is now validated normally and needs an initial-state self-loop.
|
|
40
|
+
|
|
41
|
+
Stores and mutation transactions can implement the new optional `getEntityStateSource()` method to return the state and invocation writer of the event selected by `getEntityState()`. Memory and PostgreSQL stores provide it, including transaction-scoped reads. Custom stores without the method fall back to subject-scoped `listEvents()` provenance, or to normal transition validation when neither capability is available. Unavailable provenance does not authorize the shortcut.
|
|
42
|
+
|
|
43
|
+
- f136016: Add optional Host redaction hooks for adapter output, error text, and external completion results (issue #53).
|
|
44
|
+
|
|
45
|
+
`redactAdapterOutput` runs before adapter output persistence and handoff processing. Adapter rows, success events and `extractExecutionAttestation` receive redacted output. A throwing hook or null, array, or non-object return withholds output. Host writes its reconciliation verdict before audit evidence and the succeeded adapter marker, retrying a failed idempotent verdict write once while retaining lease fencing. After a durable verdict, direct execution/recovery do not repeat the adapter. Later evidence/marker failures retain the verdict but can leave incomplete evidence; if both verdict-write attempts fail, the adapter can run again on recovery, the same exposure as a crash mid-step. Adapters behind these paths should be idempotent or carry a stable idempotency key. Lease loss can also prevent this branch from recording its verdict. A first accepted reference is retained for callback evidence.
|
|
46
|
+
|
|
47
|
+
`redactErrorText` receives exported `ErrorTextRedactionContext`. Empty error input bypasses the hook unchanged. For non-empty input, execution paths use a fixed placeholder if the hook throws or returns a non-string or empty string. Completion error-text redaction failures instead refuse before writes as non-retryable `redaction_failed`. Circuit-open and consistency messages pass through this hook as adapter text; Host's own fixed text bypasses it.
|
|
48
|
+
|
|
49
|
+
`redactExternalCompletionResult` runs before merge, validation and persistence; a throw or null, array, or non-object return refuses before writes. Both object hooks accept class instances and Dates, using the existing adapter-output `isRecord` predicate. Stored digests hash redacted result/error values when hooks are set. A matching legacy raw incoming digest is acknowledged in memory before invoking hooks; newly stored digests use redacted values. Contradiction detection compares retained values otherwise. Keep redactors deterministic and stable across deploys; a changed redactor can make a repeat look like different evidence. Existing records are not rewritten.
|
|
50
|
+
|
|
51
|
+
Consumers with an exhaustive switch over `ExternalCompletionRefusal` need the new `redaction_failed` case. `redactActionParameters` and `redactAdapterInput` behavior is unchanged. Independently of the new hooks, a worker that loses its lease inside an adapter step now rethrows `RecoverableFinalizationError` without marking the adapter failed, preserving the new owner's row. Refused handoffs write their reconciliation verdict before the adapter marker and emit safe success/accepted audit events. These are default-behavior changes.
|
|
52
|
+
|
|
53
|
+
- 4e429a3: Add `RetryableExecutionError` and `isRetryableExecutionError` to Platform's root and policies exports (issue #56). The error accepts a message and optional cause; its stable name and `EXECUTION_RETRYABLE` code support recognition across package copies.
|
|
54
|
+
|
|
55
|
+
Host adds `retryableExecutionMaxAttempts` (positive integer, default 5). During governance before policy-evaluation persistence, the error is rethrown unchanged for a leased, idempotent invocation below that cap, excluding approval resume. Workers report it through `onError`, stop the heartbeat, and can reclaim the invocation after lease expiry. Cycle results and `worker.cycle.completed` telemetry include an optional `deferred` count when positive; it counts every `RetryableExecutionError` that escapes execution, and those do not increment `failed`.
|
|
56
|
+
|
|
57
|
+
The retry region covers parameter validation, mutation-governance resolution and its single-record checkpoint write, HITL evaluation and its single-record checkpoint write, entitlement, execution authorization, policy resolution, and policy evaluation. Host reuses persisted mutation-governance and HITL decisions through checkpoint guards. The region ends before the first policy-evaluation record is written. Policy persistence, `ComplianceBlocked`, obligation listing/resolution/persistence, state-machine checks, the handler transaction, handlers, adapters, and finalization retain their existing failure/reconciliation/recovery semantics.
|
|
58
|
+
|
|
59
|
+
**Release note: failure semantics (issue #56).** Previously a transient thrown governance error terminally failed the invocation. Opting into `RetryableExecutionError` now leaves an eligible invocation `running` for lease-based recovery, with no failure/status write and no policy evidence written by that attempt. Unleased inline execution, approval resume, non-idempotent actions, and attempts at the cap fail with an explanatory message through the existing redactor. Approval resume's temporary lease does not establish that a recovery worker is configured. Obligation-resolver and policy/obligation-persistence errors remain terminal. Application callbacks should be repeatable and use stable invocation-scoped evidence. Host's Platform peer range remains `workspace:^` so published packages require the Platform release containing the new runtime exports.
|
|
60
|
+
|
|
61
|
+
- 90c1266: Scope adapter invocation reads and writes by tenant, space, and owning action invocation (issue #57). Export `AdapterInvocationScope` and add it as an optional trailing argument to store get/update methods; Host supplies it from the store-loaded invocation, and the shipped memory and PostgreSQL stores enforce all three fields when supplied. Calls without scope retain ID-only behavior for compatibility. Custom store implementers must honour the new optional `scope` argument: the two new adapter isolation checks fail for a store that ignores it, so a deployment running `runTenantIsolationChecks` against a custom store will see new failures until the store scopes these lookups. `TenantIsolationCheck["id"]` gains two members and `runTenantIsolationChecks` results now carry eight entries, so exhaustive switches over check IDs need the additional cases.
|
|
62
|
+
- 93b9830: Host now emits `StateTransitioned` after a validated bound state-machine transition and successful handler execution, following the action's `eventPhase` (issue #54).
|
|
63
|
+
|
|
64
|
+
The version-1 payload contains `entityType`, `entityId`, `fromState`, `toState`, and `causedByAction`. The subject identifies the bound entity. Host appends the fact immediately after the handler's domain events using a stable invocation-scoped event ID. With `transactionWithEvents`, it shares the domain events' transaction in the selected phase; otherwise it follows the legacy non-atomic boundary, as domain events do. It is published through the outbox by default; applications can filter it with `outbox.shouldPublish`.
|
|
65
|
+
|
|
66
|
+
Under `before_adapters`, the transition is appended during handler execution. Under `after_adapters`, it is deferred to finalization after required adapter steps succeed or accept a parked external handoff. Adapter failure withholds both domain events and the transition, so the invocation does not advance Host's recorded entity state even though handler writes may have committed. Atomic finalization commits the deferred events and invocation update together; fallback stores retain non-atomic append and update behavior. Invalid transitions, failed handlers, empty targets, undefined or empty entity IDs, and replay-only acceptance in the target state do not emit a Host transition fact.
|
|
67
|
+
|
|
68
|
+
Applications that append their own `StateTransitioned` for the bound entity get no duplicate from Host: the handler's returned events suppress Host emission when both subject type and ID match, in either event phase. Those applications should drop the workaround to use Host emission.
|
|
69
|
+
|
|
70
|
+
**Release note: validation change.** Previously, when an application did not write `toState` in its events, Host's recorded state remained unset and validation fell back to the initial state on later calls. Host's new transition fact advances recorded state, so later calls validate against that advanced state. A non-idempotent bound action that previously passed repeatedly (for example, `draft -> active`) can now be rejected on the second call (`active -> active`) unless the machine defines that edge. Model the missing edges or self-loops, or submit with the same idempotency key to deduplicate to the original invocation at submission. Applications that already emitted `toState` in their domain events see no change to this validation behavior when that `toState` matches the binding's target state; otherwise Host's fact, appended after the domain events, sets the recorded state to the target. This minor release introduces the stated failure semantics.
|
|
71
|
+
|
|
3
72
|
## 7.4.0
|
|
4
73
|
|
|
5
74
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -47,6 +47,79 @@ reclaimed so its token advances. An expired worker cannot finalize after another
|
|
|
47
47
|
`idempotencyKey` to `submitAction()` so retries resolve
|
|
48
48
|
to the original tenant-scoped invocation instead of creating another mutation.
|
|
49
49
|
|
|
50
|
+
For transient governance dependency failures, throw `RetryableExecutionError` from
|
|
51
|
+
`@fabricorg/platform` (also exported from `@fabricorg/platform/policies`). The retry region covers
|
|
52
|
+
parameter validation, mutation-governance resolution, HITL evaluation, entitlement and execution
|
|
53
|
+
authorization (including the `authorize` fallback), policy resolution, and policy evaluation. It ends
|
|
54
|
+
immediately before the first policy-evaluation record is written. Host rethrows the original error
|
|
55
|
+
without a failure/status write when:
|
|
56
|
+
|
|
57
|
+
- the invocation has a lease owner and a defined lease token;
|
|
58
|
+
- the action declares `idempotent: true`;
|
|
59
|
+
- `attemptCount` is below `retryableExecutionMaxAttempts`, a positive integer option defaulting to 5; and
|
|
60
|
+
- execution is not a `resumeApprovedInvocation()` call.
|
|
61
|
+
|
|
62
|
+
The worker reports the original error through `onError` and stops its heartbeat. Its cycle result
|
|
63
|
+
and `worker.cycle.completed` telemetry include `deferred` only when the count is greater than zero;
|
|
64
|
+
deferrals do not increment `failed`. The worker counts by error type: a `RetryableExecutionError`
|
|
65
|
+
that a store wrapper throws from Host's own reads or status writes also lands in `deferred`. In that
|
|
66
|
+
case the row likewise stays leased for a later claim, but outside this cap. Retry becomes eligible when the current lease expires, so the
|
|
67
|
+
delay is the remaining lease duration, followed by the worker's next claim; there is no immediate
|
|
68
|
+
retry or separate backoff. The next claim increments `attemptCount`. At the cap, or for an unleased,
|
|
69
|
+
non-idempotent, or approval-resume invocation, Host records `failed` with the reason retry was refused,
|
|
70
|
+
through the error-text redactor. Approval resume is excluded because its temporary lease does not
|
|
71
|
+
establish that a recovery worker is configured. Ordinary thrown errors retain terminal failure behavior.
|
|
72
|
+
|
|
73
|
+
The retry region includes Host's single-record store writes for mutation-governance and HITL
|
|
74
|
+
checkpoints. Later attempts reuse these persisted decisions through the `mutationFootprint` and
|
|
75
|
+
`hitlRoute` guards. Callbacks should be safe to repeat and return stable invocation-scoped evidence.
|
|
76
|
+
A deferral happens before the attempt writes any policy evidence: the complete definition batch must
|
|
77
|
+
evaluate before Host starts appending policy records. Policy persistence, the `ComplianceBlocked`
|
|
78
|
+
event, obligation listing/resolution/persistence, state-machine checks, the handler transaction,
|
|
79
|
+
handlers, adapters, and finalization retain their existing failure/reconciliation/recovery semantics.
|
|
80
|
+
A retryable obligation-resolver error fails terminally with the policy evidence from that attempt.
|
|
81
|
+
Preview evaluation does not enter this execution catch.
|
|
82
|
+
|
|
83
|
+
**Handler transaction reconciliation.** A handler or store transaction wrapper can throw
|
|
84
|
+
`ReconciliationRequiredError(message, { cause })` from `@fabricorg/platform` (also exported from
|
|
85
|
+
`@fabricorg/platform/policies`) when a domain effect may already have committed and an operator
|
|
86
|
+
must decide how to proceed. For example, a recovery-receipt store can check its own stored result
|
|
87
|
+
against the action's current declared events and result schema before handing that result back.
|
|
88
|
+
If its own check detects an incompatible receipt from an earlier attempt, it can throw this signal.
|
|
89
|
+
Put non-sensitive operator detail in the message: Host stores the redacted message, not `cause`.
|
|
90
|
+
The cross-copy predicate
|
|
91
|
+
`isReconciliationRequiredError` recognises the stable name and `EXECUTION_RECONCILIATION_REQUIRED`
|
|
92
|
+
code as well as local instances.
|
|
93
|
+
|
|
94
|
+
A `transactionWithEvents` wrapper cannot tell which Host transaction it is wrapping from the public
|
|
95
|
+
callback contract: Host also uses this API for `after_adapters` finalization and external completion.
|
|
96
|
+
Translate only a condition the store detects itself while recovering its own receipt, and let other
|
|
97
|
+
errors propagate unchanged. Do not match Host's internal validation-error messages; those errors
|
|
98
|
+
have no public stable code or message contract.
|
|
99
|
+
|
|
100
|
+
Host honours the signal only at the handler-transaction catch, covering `transactionWithEvents`
|
|
101
|
+
and the fallback `transaction`. It redacts the message with `redactErrorText` using `source: "handler"`,
|
|
102
|
+
records terminal `reconciliation_required`, and returns the error with the action's consistency and
|
|
103
|
+
persisted HITL route/risk tier. Status persistence emits the existing reconciliation telemetry.
|
|
104
|
+
Re-execution returns the terminal record without running the handler. Workers retain their existing
|
|
105
|
+
counting: a returned reconciliation result increments `failed`, not `completed` or `deferred`.
|
|
106
|
+
|
|
107
|
+
Atomic `transactionWithEvents` stores roll back the attempt's domain writes, domain events, and
|
|
108
|
+
Host's `StateTransitioned` together when the transaction throws. On the `transaction`-only fallback,
|
|
109
|
+
Host appends events directly. The built-in stores simply run the callback without rollback: domain
|
|
110
|
+
writes and any events Host appended during the attempt stay persisted beside the
|
|
111
|
+
`reconciliation_required` record, as they already do beside `failed`. A custom fallback transaction
|
|
112
|
+
may roll back its domain writes, but Host does not make its event appends atomic with those writes.
|
|
113
|
+
Host does not run adapters after this signal or add a new lifecycle event for it. Effects or evidence
|
|
114
|
+
committed by earlier attempts are not undone.
|
|
115
|
+
|
|
116
|
+
The signal does not change failures in pre-handler governance or validation, state-transition guard
|
|
117
|
+
exceptions, adapters, or finalization. Governance and adapter throws retain their existing `failed`
|
|
118
|
+
paths. State guards still fail closed. `after_adapters` finalization failures still escape as
|
|
119
|
+
`RecoverableFinalizationError` for recovery; other finalization throws retain their existing failure
|
|
120
|
+
path. An ordinary handler error still records `failed`. This signal requests an operator decision,
|
|
121
|
+
not an automatic retry or a claim that the current transaction committed.
|
|
122
|
+
|
|
50
123
|
New rows bind that key to the action version, actor/authority binding, and a versioned canonical
|
|
51
124
|
parameter digest. Host 3 defaults to `enforce`, which throws `IdempotencyConflictError`; configure
|
|
52
125
|
`idempotency.conflictMode` as `audit-only` for a preflight soak. Legacy null-digest rows skip only the
|
|
@@ -77,6 +150,30 @@ process, to receive breaker telemetry (`new AdapterCircuitBreakerRegistry({ onEv
|
|
|
77
150
|
manual override (`registry.get(type, operation)?.setManualOverride("force_open")`), or to supply a
|
|
78
151
|
custom `AdapterCircuitBreakerProvider`.
|
|
79
152
|
|
|
153
|
+
The default key is shared across tenants, spaces, and vendors within one registry: one tenant's
|
|
154
|
+
failures can open the breaker for another. For tenant/space/vendor isolation, inject a scoped registry:
|
|
155
|
+
|
|
156
|
+
```ts
|
|
157
|
+
import { AdapterCircuitBreakerRegistry, scopedAdapterCircuitBreakerKey } from "@fabricorg/platform/adapters";
|
|
158
|
+
|
|
159
|
+
const adapterCircuitBreakers = new AdapterCircuitBreakerRegistry({ key: scopedAdapterCircuitBreakerKey });
|
|
160
|
+
// Pass adapterCircuitBreakers to createGovernedActionHost({ store, registry, adapters, ... }).
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Host passes the trusted `{ tenantId, spaceId }` invocation context to `forAdapter(adapter, scope)`;
|
|
164
|
+
providers receive it from `@fabricorg/platform-host` 7.5.0 on. Existing one-argument providers remain compatible. The first config
|
|
165
|
+
seen for a registry key wins; later conflicting configs reuse the existing breaker without throwing.
|
|
166
|
+
Adapters sharing a key should use the same configuration. The helper URI-encodes components and
|
|
167
|
+
preserves a readable `adapterType:operation` suffix for ordinary identifiers. Missing space or vendor
|
|
168
|
+
is distinct from an empty string; `vendor` is required by the TypeScript adapter contract.
|
|
169
|
+
Calls made outside Host without scope use a separate vendor-specific `unscoped` namespace and share
|
|
170
|
+
breakers there; tenant isolation requires scope. Use non-secret identifiers because keys appear in
|
|
171
|
+
telemetry and errors. Breakers are never evicted: a scoped key creates one breaker per tenant, space, vendor and operation
|
|
172
|
+
for the life of the registry, and those keys are high-cardinality, so do not use them as metric labels.
|
|
173
|
+
For a custom-keyed breaker, use
|
|
174
|
+
`registry.getByKey(scopedAdapterCircuitBreakerKey(adapter, scope))`; `get(type, operation)` continues
|
|
175
|
+
to look up the default key.
|
|
176
|
+
|
|
80
177
|
The breaker is consulted immediately before every attempt, including retries. Host writes the attempt number to the adapter invocation record first, in `prepareAttempt`, so nothing awaits between admission and the adapter call; a store failure there is not counted against the breaker. In `monitor` mode it
|
|
81
178
|
never blocks. In `enforce` mode, while it is open, Host does not call the adapter: the adapter
|
|
82
179
|
invocation is marked `failed`, `AdapterInvocationFailed` is appended with
|
|
@@ -153,17 +250,26 @@ in action parameters or workflow history.
|
|
|
153
250
|
Production stores can implement `AtomicMutationPlatformHostStore.transactionWithEvents`. The
|
|
154
251
|
transaction-scoped `db`, event sequence, event append, event listing, and invocation update methods
|
|
155
252
|
must all use the same database transaction.
|
|
253
|
+
The optional transaction-scoped `updateAdapterInvocation` and `appendExternalReconciliation`
|
|
254
|
+
operations extend this unit of work to degraded adapter finalization, as described under redaction
|
|
255
|
+
below.
|
|
156
256
|
|
|
157
257
|
`PostgresPlatformHostStore` enables this capability when constructed with a
|
|
158
258
|
`PostgresPlatformHostTransactionProvider`. The application owns that narrow binder because only the
|
|
159
259
|
application knows how its `TDb` is rebound to the transaction's SQL client.
|
|
160
260
|
|
|
161
|
-
|
|
162
|
-
|
|
261
|
+
With `transactionWithEvents`:
|
|
262
|
+
|
|
263
|
+
- `before_adapters`: handler domain writes, declared domain events, and the Host-generated
|
|
264
|
+
`StateTransitioned` fact commit or roll back together.
|
|
265
|
+
- `after_adapters`: declared domain events, the Host-generated `StateTransitioned` fact, and
|
|
266
|
+
invocation finalization commit together after required adapter steps succeed or accept a
|
|
267
|
+
parked external handoff. Handler writes have already committed separately. A finalization
|
|
163
268
|
failure leaves the invocation recoverable; succeeded adapter checkpoints are not repeated.
|
|
164
269
|
|
|
165
|
-
Stores without
|
|
166
|
-
|
|
270
|
+
Stores without `transactionWithEvents` retain the legacy non-atomic boundary for domain writes,
|
|
271
|
+
events, and invocation persistence. A partial append or finalization failure can leave some
|
|
272
|
+
records persisted; this path does not provide atomic domain-write/event persistence.
|
|
167
273
|
|
|
168
274
|
`createDurableSagaParentLifecycle()` requires this capability. Each parent lifecycle transition uses a
|
|
169
275
|
stable `lifecycleId` as its causation identity, appends its audit event, and updates the parent invocation
|
|
@@ -229,6 +335,80 @@ and pass an opaque identifier instead; action parameters are intentionally durab
|
|
|
229
335
|
Hosts should additionally configure `redactActionParameters` as a fail-safe allowlist for actions
|
|
230
336
|
whose ingress is adjacent to secret material; redaction runs before the invocation row is created.
|
|
231
337
|
|
|
338
|
+
Three optional `PlatformHostOptions` hooks cover other Host-recorded values.
|
|
339
|
+
|
|
340
|
+
| Hook | When it runs and what uses its return value |
|
|
341
|
+
| --- | --- |
|
|
342
|
+
| `redactAdapterOutput(actionId, adapterType, operation, output)` | Once after a successful adapter result is converted to output, before handoff handling. The adapter row, `AdapterInvocationSucceeded` payload, and `extractExecutionAttestation` input use the redacted output. |
|
|
343
|
+
| `redactErrorText(context, message)` | At the origin of caught handler/execution errors, adapter result errors and ambiguous-outcome reasons, and completion errors. The invocation/adapter errors, reconciliation messages and reasons, completion evidence, failure events, and returned error reuse that text. |
|
|
344
|
+
| `redactExternalCompletionResult(actionId, provider, result)` | After a completion matches its invocation, before merging, result-schema validation, persistence, and digesting. See [External completion](#external-completion). |
|
|
345
|
+
|
|
346
|
+
`ErrorTextRedactionContext` is exported. It contains `actionId` and `source` (`handler`, `adapter`,
|
|
347
|
+
`external_completion`, or `execution`); adapter errors also supply `adapterType`, `operation`, and
|
|
348
|
+
`provider` (the adapter vendor), while completion errors supply `provider`. Exceptions from
|
|
349
|
+
application execution hooks such as `getInput` and `extractExecutionAttestation` use `execution`.
|
|
350
|
+
Host-authored fixed messages and structured identifiers, including `externalReference`, bypass this
|
|
351
|
+
hook. Circuit-open errors and consistency errors use adapter context because an adapter can supply
|
|
352
|
+
those messages or their interpolated values. When Host adds context around a caught store error,
|
|
353
|
+
only the caught part is redacted. Empty originating error text bypasses the hook unchanged. For non-empty input, a hook that throws,
|
|
354
|
+
returns a non-string, or returns an empty string uses `Error text withheld: the error-text redactor failed.` on execution paths. These paths retain
|
|
355
|
+
their failure classification when the error can be persisted. For a completion error, the same hook
|
|
356
|
+
failures refuse with non-retryable `redaction_failed` before writes, allowing a sender to submit the
|
|
357
|
+
callback after the redactor is repaired.
|
|
358
|
+
|
|
359
|
+
If `redactAdapterOutput` throws or returns null, an array, or a non-object value, the external effect
|
|
360
|
+
has already happened. Host withholds output and records a `reconciliation_required` verdict. Its
|
|
361
|
+
safe evidence includes the retained handoff's `ExternalOperationAccepted`,
|
|
362
|
+
`AdapterInvocationSucceeded` with redacted input and `outputWithheld: true` without output, and
|
|
363
|
+
fixed-text reconciliation evidence when the store supports governance. The events use the normal
|
|
364
|
+
step deduplication keys. A first accepted reference is retained as `pendingCompletion`, so its
|
|
365
|
+
callback can still land as evidence. References come from the original adapter result.
|
|
366
|
+
|
|
367
|
+
For withheld output and refused handoffs (immediate completion or a second handoff), Host uses
|
|
368
|
+
`transactionWithEvents` when the transaction supplies `updateAdapterInvocation`,
|
|
369
|
+
`updateLeasedActionInvocation` for a leased invocation, and `appendExternalReconciliation` for
|
|
370
|
+
withheld output on a governance-capable store. These optional members must
|
|
371
|
+
share the event/verdict transaction; the adapter update must enforce the store method's supplied
|
|
372
|
+
tenant, space, and invocation scope. Both built-in stores provide them when configured with a
|
|
373
|
+
transaction provider. Custom stores must implement them to get this atomic behavior.
|
|
374
|
+
|
|
375
|
+
When atomic finalization succeeds, evidence events, any withheld-output reconciliation entry, the adapter's `succeeded`
|
|
376
|
+
marker, and the terminal `reconciliation_required` verdict commit together. Host writes evidence and
|
|
377
|
+
the marker before the fenced verdict, which releases the lease, then emits status telemetry after
|
|
378
|
+
commit. It retries the whole transaction once on a non-lease error. If the retry also fails with a
|
|
379
|
+
non-lease error, Host selects the verdict-first fallback described below. Lease loss is not retried
|
|
380
|
+
or converted to fallback; it raises `RecoverableFinalizationError`. Atomic rollback leaves the
|
|
381
|
+
invocation and adapter row `running` until a retry or fallback commits. If lease loss or an unavailable
|
|
382
|
+
fallback verdict prevents finalization, the invocation remains reclaimable and the adapter can run
|
|
383
|
+
again under the same `adapterInvocationId` idempotency key. Adapters should be idempotent or carry
|
|
384
|
+
that stable idempotency key.
|
|
385
|
+
|
|
386
|
+
Without the required transaction members, or after two non-lease transaction failures, Host keeps
|
|
387
|
+
the verdict-first order: the `reconciliation_required` verdict precedes evidence and the adapter marker. It retries the verdict
|
|
388
|
+
write once, except on lease loss. A later evidence or marker failure raises
|
|
389
|
+
`RecoverableFinalizationError` after the verdict commits and can leave a terminal verdict with missing
|
|
390
|
+
evidence, including when the same deterministic failure affects the transaction and fallback. If both verdict-write attempts fail, the adapter can run again on recovery,
|
|
391
|
+
the same exposure as a crash mid-step; adapters should be idempotent or carry a stable idempotency key.
|
|
392
|
+
Direct execution returns a durable terminal verdict and recovery excludes it through terminal-status
|
|
393
|
+
checks. Refused-handoff completion reconciliation logging stays after finalization. After atomic
|
|
394
|
+
finalization, a failed log write leaves the committed verdict and adapter marker intact if the
|
|
395
|
+
unfenced invocation-error update succeeds. If that error update also fails, the existing adapter-error
|
|
396
|
+
catch can mark the adapter and invocation `failed`.
|
|
397
|
+
|
|
398
|
+
`extractExecutionAttestation` receives the redacted output, so an extractor that needs a stripped
|
|
399
|
+
field can fail after the external effect, following the existing extractor-failure behavior.
|
|
400
|
+
|
|
401
|
+
`redactActionParameters` and `redactAdapterInput` behavior is unchanged. Independently of hook
|
|
402
|
+
configuration, a stale worker that loses its lease inside an adapter step rethrows without marking
|
|
403
|
+
the adapter failed, preserving the new owner's row. Refused-handoff ordering and audit events also
|
|
404
|
+
change without the new hooks.
|
|
405
|
+
|
|
406
|
+
Applications own the safety of domain events/results, policy and HITL evidence, explicit
|
|
407
|
+
attestations/reconciliations, saga lifecycle inputs, approval reasons, and outbox publisher errors.
|
|
408
|
+
Parameter-schema issue messages used for `validation_failed`, the `HITL rejected` reason, policy
|
|
409
|
+
block reasons, and action-result validation paths are not passed through `redactErrorText`.
|
|
410
|
+
These hooks do not rewrite historical records or exceptions delivered to a worker's `onError`.
|
|
411
|
+
|
|
232
412
|
## Submission snapshot
|
|
233
413
|
|
|
234
414
|
`submitAction` reads the scalar fields of the caller's `SubmitActionInput` once, before its first
|
|
@@ -393,13 +573,104 @@ controlled deployment migration step, and hydrate projections from `listEvents()
|
|
|
393
573
|
uses ordered immutable checksums, a migration ledger, and a transaction-scoped advisory lock. It
|
|
394
574
|
remains safe to call at startup, but deployment-owned migrations are preferred.
|
|
395
575
|
|
|
396
|
-
State-machine transition checks run inside the Host mutation unit of work.
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
576
|
+
State-machine transition checks run inside the Host mutation unit of work, before the handler. Stores implementing `transactionWithEvents`
|
|
577
|
+
should expose transaction-scoped `getEntityState()` so the guard reads the same database snapshot
|
|
578
|
+
used by the handler. Existing custom stores without that optional transaction method fall back
|
|
579
|
+
to the store-level authoritative read.
|
|
580
|
+
|
|
581
|
+
An idempotent action may resume in its target state despite a missing self-loop only when the
|
|
582
|
+
event that set the current recorded state belongs to that same `actionInvocationId`. The handler
|
|
583
|
+
can rerun while stable event IDs suppress duplicate events. A new invocation validates normally;
|
|
584
|
+
model a self-loop if it should re-apply the state, or use the same idempotency key to deduplicate
|
|
585
|
+
to the original invocation at submission.
|
|
586
|
+
|
|
587
|
+
The optional `getEntityStateSource(tenantId, spaceId, entityType, entityId)` store and transaction
|
|
588
|
+
method returns `{ state, actionInvocationId }` from the same event as `getEntityState()`, or
|
|
589
|
+
`undefined` when provenance is unavailable. Memory transaction reads include pending events.
|
|
590
|
+
Host prefers the transaction method, then the store method. If neither is implemented, it uses
|
|
591
|
+
transaction `listEvents()`, otherwise store `listEvents()` when available, selecting the subject's
|
|
592
|
+
highest-sequence event with a string `payload.toState`. The source state and invocation writer
|
|
593
|
+
must match the current state and invocation. Missing provenance leaves normal validation in force;
|
|
594
|
+
custom stores can continue compiling without the optional method.
|
|
595
|
+
|
|
596
|
+
After a valid transition for a binding with a non-empty entity ID and target, Host appends
|
|
597
|
+
`StateTransitioned` immediately after the handler's domain events, following the action's
|
|
598
|
+
`eventPhase`. Its subject is the bound entity and its version-1 payload is
|
|
599
|
+
`{ entityType, entityId, fromState, toState, causedByAction }`. `fromState` is the trusted state
|
|
600
|
+
read before the handler, and `causedByAction` is the action ID. The event uses a stable
|
|
601
|
+
invocation-scoped ID and is publishable through the outbox by default; `outbox.shouldPublish`
|
|
602
|
+
can filter it. It does not need an `emitsEvents` declaration from the action.
|
|
603
|
+
|
|
604
|
+
For `before_adapters`, the transition is appended with the handler's domain events. For
|
|
605
|
+
`after_adapters`, it is deferred to finalization after required adapter steps succeed or
|
|
606
|
+
accept a parked external handoff, immediately after domain events. Adapter failure withholds
|
|
607
|
+
both domain events and the transition, leaving the built-in stores' ledger-derived entity state
|
|
608
|
+
unchanged by that invocation even though handler writes may have committed. Custom stores that
|
|
609
|
+
read domain rows can already observe the target state at this point. With `transactionWithEvents`, the
|
|
610
|
+
transition shares the transaction used for domain events in the selected phase; otherwise it
|
|
611
|
+
follows the same legacy non-atomic boundary as domain events.
|
|
612
|
+
|
|
613
|
+
For idempotent `after_adapters` actions, Host saves `pendingStateTransition` on the invocation.
|
|
614
|
+
It contains `{ entityType, entityId, fromState, toState }` from the validated transition.
|
|
615
|
+
Custom stores must persist and return this optional invocation-record field through
|
|
616
|
+
`updateActionInvocation`, including the transaction-scoped method. With a mutation transaction,
|
|
617
|
+
Host writes the field after the handler succeeds and result/events validate, just before returning
|
|
618
|
+
from the transaction callback. This keeps the invocation row lock out of the handler run while the
|
|
619
|
+
saved transition commits or rolls back with the domain writes. The `transaction`-only fallback saves
|
|
620
|
+
it separately before the handler. Non-idempotent actions do not write this field. It is retained as
|
|
621
|
+
evidence after completion. PostgreSQL `ensureSchema()` adds `pending_state_transition jsonb`.
|
|
622
|
+
|
|
623
|
+
On an admitted same-invocation replay, Host uses the saved `fromState` when the saved entity type,
|
|
624
|
+
entity ID, and target match the current binding and state. The fact follows the usual event phase
|
|
625
|
+
and stable `state:transition` ID, so built-in stores deduplicate an already appended event.
|
|
626
|
+
Invocations started before upgrading, or stores that omit this evidence, retain replay acceptance
|
|
627
|
+
without a re-emitted transition. Mismatched evidence also suppresses replay emission.
|
|
628
|
+
|
|
629
|
+
Invalid transitions, failed handlers, replay acceptance without matching saved evidence, and
|
|
630
|
+
bindings without a non-empty entity ID or target do not produce a Host transition fact. If the handler
|
|
631
|
+
returns its own `StateTransitioned` with the bound entity's subject type and ID, Host suppresses
|
|
632
|
+
its generated fact in either event phase. Applications using that workaround should remove it
|
|
633
|
+
to use Host emission.
|
|
634
|
+
|
|
635
|
+
Host's new transition fact advances recorded state even if handler domain events omit `toState`.
|
|
636
|
+
Later calls validate against that advanced state; a new invocation can fail if the
|
|
637
|
+
machine lacks the corresponding edge or self-loop. Model those edges, or submit with the same
|
|
638
|
+
idempotency key to deduplicate to the original invocation. Applications already recording `toState` in
|
|
639
|
+
domain events retain their existing state advancement behavior when that `toState` matches the
|
|
640
|
+
binding's target; otherwise Host's later fact sets the recorded state to the target.
|
|
400
641
|
|
|
401
642
|
## External completion
|
|
402
643
|
|
|
644
|
+
Configure `redactExternalCompletionResult` to withhold sensitive completion fields. Host applies it
|
|
645
|
+
once to a supplied result (including a result attached to a failed callback). The redacted value
|
|
646
|
+
feeds `externalCompletion.result`; for a completed outcome it also feeds the adapter output,
|
|
647
|
+
`ExternalOperationCompleted` payload, merged invocation result, and returned result. A supplied
|
|
648
|
+
`completion.error` passes through `redactErrorText` before recording, including when attached to a
|
|
649
|
+
completed outcome. A missing error uses Host's fixed failure message, which bypasses the hook.
|
|
650
|
+
|
|
651
|
+
The recorded digest hashes the redacted result and error when hooks are configured. A repeat first compares the raw incoming digest in memory before invoking any hook, recognizing
|
|
652
|
+
legacy records even if a newly configured redactor would fail. Otherwise it compares the redacted digest. Newly stored digests use the redacted representation.
|
|
653
|
+
Contradiction detection therefore compares what redactors retain; differences they discard are
|
|
654
|
+
considered the same news. The compatibility match leaves historical records intact.
|
|
655
|
+
|
|
656
|
+
Redactors should be deterministic and stable across deploys. A changed redactor can make a repeat
|
|
657
|
+
look like different evidence and trigger contradiction handling.
|
|
658
|
+
|
|
659
|
+
The object hooks accept non-null, non-array objects, including class instances and Dates, matching
|
|
660
|
+
the existing adapter-output `isRecord` predicate. Strings, arrays and null are rejected.
|
|
661
|
+
|
|
662
|
+
If completion-result redaction throws or returns null, an array, or a non-object, or completion
|
|
663
|
+
error-text redaction throws or returns a non-string or empty string for non-empty input, `ExternalCompletionError.refusal` is `redaction_failed` and
|
|
664
|
+
`retryable` is false. Host refuses it before writing the invocation, adapter row, event, or
|
|
665
|
+
reconciliation evidence; fix the redactor before submitting again. Consumers with exhaustive
|
|
666
|
+
`ExternalCompletionRefusal` switches need a `redaction_failed` case. Other permanent refusals are
|
|
667
|
+
`not_found`, `not_awaiting`, `immediate_contract`, `reference_mismatch`, `provider_mismatch`,
|
|
668
|
+
`invalid_completion`, and `result_invalid`; `still_executing` is retryable.
|
|
669
|
+
|
|
670
|
+
While a redaction hook is failing, a contradicting repeat that needs redaction is also refused with
|
|
671
|
+
`redaction_failed`, without recording a contradiction. A matching legacy raw digest still returns
|
|
672
|
+
before hooks run. Empty completion error input bypasses the error-text hook unchanged.
|
|
673
|
+
|
|
403
674
|
An action declaring `completion: "accepted"` or `"long-running"` may hand its work to an external
|
|
404
675
|
system. The adapter returns `acceptedExternalOperation: { externalReference }`; the Host parks the
|
|
405
676
|
invocation as `running` with no lease, emits `ExternalOperationAccepted`, and leaves it for the
|
|
@@ -421,7 +692,8 @@ reference, outcome, result, error, and evidence. A contradictory one, a flipped
|
|
|
421
692
|
outcome with different evidence, moves the invocation to
|
|
422
693
|
`reconciliation_required`, keeps the first outcome on record, logs the second, and emits
|
|
423
694
|
`ExternalOperationContradicted`. Under an atomic store the read and write happen under one lock. The
|
|
424
|
-
callback's result is
|
|
695
|
+
callback's redacted result is merged with the invocation result, stripped of private fields, and
|
|
696
|
+
validated against the action's result schema before writes. The stored merge is what is validated.
|
|
425
697
|
An invocation that ended for another reason while a handoff was pending keeps its status and records
|
|
426
698
|
the callback as evidence. Declare `completionDeadlineMs` on `long-running` actions;
|
|
427
699
|
`reconcileOverdueCompletions()` moves overdue handoffs to reconciliation while keeping the handoff so
|
|
@@ -453,12 +725,20 @@ cast in an index expression is not immutable and PostgreSQL refuses it.
|
|
|
453
725
|
|
|
454
726
|
## Tenant isolation
|
|
455
727
|
|
|
456
|
-
`tenantIsolationChecks()` and `runTenantIsolationChecks(subject)`
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
728
|
+
`tenantIsolationChecks()` and `runTenantIsolationChecks(subject)` run eight checks against a host
|
|
729
|
+
and its store. They prove scope on the invocation reads and writes, idempotency keys, event streams,
|
|
730
|
+
worker claims, external completion, an optional governed read, and adapter checkpoint reads and
|
|
731
|
+
writes they probe. The subject supplies a host, a recoverable store, a registered action, and two
|
|
732
|
+
scopes in different tenants. Adapter checks create probes in dedicated spaces and retire them afterwards.
|
|
733
|
+
|
|
734
|
+
Host passes `AdapterInvocationScope` from the store-loaded action invocation to
|
|
735
|
+
`getAdapterInvocation(id, scope)` and `updateAdapterInvocation(id, patch, scope)`. Custom stores
|
|
736
|
+
must match `tenantId`, `spaceId`, and `actionInvocationId` when scope is supplied. A mismatch returns
|
|
737
|
+
`undefined` on read and leaves the row unchanged on update (a not-found error is permitted).
|
|
738
|
+
The trailing argument is optional to preserve existing custom-store and direct-call compatibility;
|
|
739
|
+
calls without it retain ID-only behavior in the shipped stores. The two adapter isolation checks
|
|
740
|
+
fail for stores that ignore scope. Deployments running `runTenantIsolationChecks` against a custom
|
|
741
|
+
store will see new failures until that store honours the argument.
|
|
462
742
|
|
|
463
743
|
## Runtime observability and health
|
|
464
744
|
|