@synapsor/runner 1.1.1 → 1.4.1

package/docs/rfcs/005-bounded-set-writeback.md

@@ -0,0 +1,164 @@
+# RFC 005: Bounded Set Writeback
+
+Status: implementation gate
+
+## Purpose
+
+Runner 1.3 may execute a reviewed set write only when the set is fixed by the
+contract, frozen before approval, bounded by both row count and business value,
+and applied atomically. A row cap by itself is not a safety boundary.
+
+This RFC authorizes three direct operations:
+
+- fixed-predicate set UPDATE;
+- fixed-predicate set DELETE;
+- exact-review batch INSERT.
+
+Free-form predicates, model-selected columns or operators, unbounded sets,
+UPSERT, DDL, cross-table work, and external effects remain app-owned executor
+work.
+
+## Canonical Model
+
+Set operations are explicit. They never reuse a single-row lookup by merely
+raising `max_rows`.
+
+```json
+{
+  "operation": {
+    "kind": "update",
+    "cardinality": "set",
+    "selection": {
+      "all": [
+        { "column": "status", "operator": "eq", "value": "overdue" }
+      ]
+    },
+    "max_rows": 10,
+    "aggregate_bounds": [
+      { "column": "balance_cents", "measure": "before", "maximum": 50000 }
+    ]
+  }
+}
+```
+
+The first release accepts only contract literals in `selection`. The typed AST
+reserves no raw SQL escape hatch. The model can trigger the reviewed capability
+but cannot choose a predicate, column, operator, ordering, range, wildcard, or
+set size.
+
+Batch INSERT uses an explicitly typed bounded item-array argument. Every item
+is fully materialized in the proposal, receives the trusted tenant, and has a
+source-enforced unique dedup identity. A generic JSON blob is not accepted.
+
+## Hard Limits
+
+- `MAX ROWS` is mandatory and must be between 1 and 100.
+- at least one aggregate value bound is mandatory;
+- a proposal query fetches at most `MAX ROWS + 1` rows;
+- at most 64 reviewed columns may be captured per row;
+- before/after payloads and digests are bounded before persistence;
+- overflow rejects the proposal; it is never truncated to the cap.
+
+The implementation ceiling of 100 is a safety limit, not a throughput claim.
+
+## Frozen Set
+
+At proposal time Runner stores, in deterministic primary-key order:
+
+- exact primary key;
+- trusted tenant;
+- expected version for UPDATE/DELETE;
+- allowlisted before and after values;
+- per-row before and after digests;
+- count, aggregate measurements, and one set digest.
+
+Apply never re-runs the broad selection predicate. It locks the frozen primary
+keys in the same deterministic order, rechecks tenant and every expected
+version, and mutates the complete set in one transaction. A missing or stale
+member rolls back the whole transaction.
+
+## Approval
+
+All set operations require verified human/operator approval in 1.3. Policy
+auto-approval is rejected rather than silently downgraded. Set hard DELETE and
+batch INSERT are never auto-approved. This keeps row/value-aware policy design
+out of the first release.
+
+## Exact Effects
+
+The receipt and replay carry every affected primary key and bounded before/
+after digest. Kept-out fields are never fetched merely to improve audit detail.
+The source receipt and mutation share one transaction in `source_db` mode.
+`runner_ledger` mode retains the documented post-source-commit ambiguity and
+uses `reconciliation_required`; it never retries an ambiguous batch.
+
+## Operation Rules
+
+### Set UPDATE
+
+- fixed literal predicate;
+- fixed or scalar-argument patch with existing per-field bounds;
+- exact version guard and monotonic version advance for every row;
+- mandatory aggregate bound;
+- one transaction and exact affected-row count.
+
+### Set DELETE
+
+- fixed literal predicate;
+- exact version guard for every row;
+- verified human approval;
+- no write triggers or cascading foreign keys;
+- bounded tombstones and exact identities.
+
+Soft delete remains the preferred set operation because it is a guarded
+UPDATE and can later be reversed safely.
+
+### Batch INSERT
+
+- one typed item array with `max_items` no greater than `MAX ROWS`;
+- every candidate row shown in the proposal;
+- trusted tenant injected by Runner;
+- source UNIQUE/PRIMARY KEY dedup identity for every item;
+- per-item bounds plus a mandatory aggregate bound;
+- all rows and source receipts commit atomically or none do.
+
+## Threat Cases And Fixtures
+
+| Threat | Mitigation | Conformance fixture |
+| --- | --- | --- |
+| R1 selection abuse | literal typed predicate; no model predicate inputs | `bounded-set-r1-fixed-selection` |
+| R2 misleading cap | row cap plus aggregate value bound; overflow rejects | `bounded-set-r2-count-value-cap` |
+| R3 TOCTOU drift | frozen PK/version set; apply never broad-requeries | `bounded-set-r3-frozen-drift` |
+| R4 partial application | one transaction; exact count; rollback on any member | `bounded-set-r4-atomic-rollback` |
+| R5 audit collapse | every key and bounded before/after digest in receipt/replay | `bounded-set-r5-exact-receipt` |
+| R6 DELETE cascades | hard DELETE preflight rejects cascades/write triggers | `bounded-set-r6-delete-side-effects` |
+| R7 auto-approval amplification | human approval mandatory; policy mode rejected | `bounded-set-r7-human-approval` |
+
+Each fixture must be exercised by canonical validation and runtime/adapter
+tests. A fixture file that is never loaded by a test is not evidence.
+
+## Crash And Receipt Matrix
+
+| Outcome | `source_db` | `runner_ledger` |
+| --- | --- | --- |
+| before source BEGIN | no effect | durable intent, no effect |
+| one member stale | complete rollback and conflict receipt | complete rollback and terminal conflict |
+| process death before source commit | complete rollback | complete rollback; intent may be safely resolved as not committed |
+| process death after source commit | source mutation and receipt prove outcome atomically | `reconciliation_required`; never automatic retry |
+| retry after completed receipt | `already_applied` | `already_applied` from authoritative ledger |
+
+No mode claims a distributed transaction across Runner's ledger and an external
+source database.
+
+## Release Gate
+
+The feature remains unavailable until Postgres and MySQL live tests prove:
+
+- cap+1 and aggregate overflow reject before proposal persistence;
+- one stale member and injected failure leave every source row unchanged;
+- lock order is deterministic;
+- exact IDs/digests appear in receipts and replay;
+- hard DELETE side effects and auto-approval are rejected;
+- batch INSERT dedup and crash behavior are safe in both receipt modes;
+- seeded 1, 10, and 100 row measurements are recorded as local evidence only.
+

package/docs/rfcs/006-reviewed-reversible-change-sets.md

@@ -0,0 +1,152 @@
+# RFC 006: Reviewed Reversible Change Sets
+
+Status: implemented and verified for the prepared Runner 1.4 release line
+
+## Problem
+
+An applied direct database write currently has evidence, approval, a guarded
+writeback receipt, and replay, but no bounded compensating action. Correcting a
+mistake requires an operator to author a new forward proposal manually.
+
+Runner should be able to prepare a reviewed compensation without becoming a
+database time-travel system or introducing an unreviewed rollback path.
+
+## Safety boundary
+
+Reversal is opt-in contract authority:
+
+```json
+{
+  "reversibility": {
+    "mode": "reviewed_inverse"
+  }
+}
+```
+
+The declaration is valid only for Runner-owned `direct_sql` writeback. It does
+not apply to app handlers, Cloud workers, email, payments, or any other
+external effect.
+
+`synapsor-runner revert <proposal-id>` creates a new proposal. It never:
+
+- mutates the source database;
+- approves the proposal;
+- bypasses tenant or operator identity checks;
+- appears in the model-facing MCP tool list.
+
+The new proposal must be independently approved and applied through the normal
+trusted operator path.
+
+## Protocol split
+
+Published v1, v2, and v3 forward writeback jobs keep their existing meaning.
+Reversible forward jobs carry an additive, explicit inverse-capture request.
+The resulting execution receipt carries a bounded inverse descriptor only
+after a successful, unambiguous source transaction.
+
+Compensation is executed through a distinct normalized protocol v4 job. A v4
+job contains:
+
+- the exact source, tenant, and primary-key authority;
+- a bounded, deterministically ordered member list;
+- the original single-row or set cardinality;
+- the state expected after the forward write;
+- only the reviewed values to restore or remove;
+- the original row and aggregate caps;
+- the original approval role and quorum;
+- lineage linking root, parent, and reverted proposals and the forward receipt;
+- a maximum lineage depth.
+
+This prevents compensation semantics from being inferred from a legacy job.
+
+## Operation semantics
+
+### UPDATE and soft delete
+
+The inverse restores only previously reviewed writable values. The row must
+still match the exact reviewed state left by the forward write. A monotonic
+version column advances again during compensation; it is never decremented.
+Any intervening write fails closed.
+
+A soft delete is an UPDATE of a reviewed status or deletion marker and follows
+these same rules.
+
+### INSERT
+
+The inverse deletes only the exact Runner-created identity. The current row
+must still match the allowlisted values inserted by Runner. Trigger and foreign
+key safety is rechecked immediately before deletion. A new dependent row or a
+changed value blocks compensation.
+
+Because this delete removes a row whose complete Runner-supplied values are
+known, its own inverse may reinsert that exact row. It cannot infer or restore
+unreviewed database-generated or trigger-generated state.
+
+### Hard DELETE
+
+A general hard DELETE records bounded reviewed before-state evidence but is
+not automatically restorable. Generated identities, hidden required columns,
+cascaded children, and trigger side effects cannot be reconstructed safely.
+The receipt marks the inverse unavailable with specific reason codes and the
+CLI recommends soft delete or an app-owned compensation capability.
+
+## Bounded sets
+
+Set compensation uses exactly the original frozen identities. It cannot widen
+the predicate or increase the original row/value caps. Every member is locked
+in deterministic primary-key order and checked before any mutation. One stale
+member rolls back the entire transaction.
+
+The inverse descriptor is bounded to 100 members and 256 scalar fields per
+member, matching the public protocol hard ceilings.
+
+## Receipt authority and crash behavior
+
+With source-database receipt authority, mutation and source receipt remain in
+one transaction. The inverse descriptor is deterministic from the immutable
+job plus the state verified in that transaction, so an idempotent retry can
+reproduce it without rereading kept-out columns.
+
+With Runner-ledger authority, the existing cross-database crash window remains.
+An ambiguous forward or compensation outcome enters
+`reconciliation_required`. Runner does not generate or apply a revert until an
+operator reconciles the original outcome and an applied receipt with an
+available inverse exists.
+
+## Lineage
+
+Every compensation proposal records:
+
+- `root_proposal_id`;
+- `parent_proposal_id`;
+- `reverts_proposal_id`;
+- `forward_receipt_hash`;
+- `depth`.
+
+Depth is capped at 16. The same applied receipt may have only one active or
+completed compensation child. To undo a compensation, the operator reverts the
+child proposal, creating a linear audited chain instead of a cycle.
+
+## Data minimization
+
+Inverse capture may retain only:
+
+- fixed primary-key and trusted tenant identity;
+- conflict/version metadata;
+- fields in the reviewed write allowlist;
+- Runner-supplied deterministic deduplication fields needed to identify an
+  inserted row.
+
+Visible read fields that were not writable and all kept-out fields are excluded.
+Reversibility is rejected when the safe inverse cannot be represented with that
+bounded data.
+
+## Explicit non-goals
+
+- point-in-time recovery or WAL replacement;
+- automatic rollback;
+- model-facing revert, approve, apply, or receipt tools;
+- restoration of arbitrary hard deletes;
+- compensation of app-owned executors or external effects;
+- cross-table, cross-database, or unbounded compensation;
+- weakening the existing optimistic concurrency and receipt-authority model.

package/docs/runner-config-reference.md

@@ -112,7 +112,13 @@ not print database URLs or create the schema.
       "read_url_env": "BILLING_POSTGRES_READ_URL",
       "write_url_env": "BILLING_POSTGRES_WRITE_URL",
       "read_only": false,
-      "statement_timeout_ms": 3000
+      "statement_timeout_ms": 3000,
+      "receipts": {
+        "authority": "source_db",
+        "provisioning": "precreated",
+        "schema": "synapsor",
+        "table": "writeback_receipts"
+      }
     }
   }
 }
@@ -121,8 +127,13 @@ not print database URLs or create the schema.
 `engine` is `postgres` or `mysql`. `read_url_env` is required. Use a
 least-privilege read credential. `write_url_env` is optional unless direct SQL
 writeback must apply; it should name a separate restricted writer. `read_only:
-true` forbids direct SQL writeback. `statement_timeout_ms` is a positive read
-timeout. `ssl` carries adapter-specific reviewed SSL options when used.
+true` forbids direct SQL writeback. `statement_timeout_ms` is an
+operator-controlled positive timeout used for reads and direct SQL writeback.
+PostgreSQL applies it as transaction-local `statement_timeout` and
+`lock_timeout`. MySQL applies it to read/preflight execution and rounds it up
+to whole seconds for `innodb_lock_wait_timeout`; MySQL does not provide the
+same general DML statement-timeout guarantee as PostgreSQL. `ssl` carries
+adapter-specific reviewed SSL options when used.
 
 A contract's `SOURCE billing_postgres` must exactly match a `sources` key.
 
@@ -131,8 +142,31 @@ Long-running servers reuse native driver pools. Optional `pool` keys are
 `idle_timeout_ms` (30000), `queue_timeout_ms` (5000), and `queue_limit`
 (default `max(10, max_connections * 4)`). Queue overflow returns
 `SOURCE_POOL_QUEUE_FULL`; acquisition timeout returns `SOURCE_POOL_TIMEOUT`.
+With result envelope v2, these conditions and recognized transient driver
+connection failures are model-facing `TEMPORARILY_UNAVAILABLE` errors with
+`retryable: true` and a bounded `retry_after_ms`. Operational logs retain only
+a normalized safe runtime code, not the driver message, host, or credentials.
 One-shot CLI commands close their pools.
 
+### Source receipts
+
+`sources.<name>.receipts.authority` is `source_db` or `runner_ledger`.
+
+- `source_db` requires `provisioning: "precreated"` or `"auto_migrate"`.
+  Optional fixed `schema` and `table` identifiers select the receipt table.
+  The source mutation and receipt share one transaction.
+- `runner_ledger` forbids `provisioning`, `schema`, and `table`. It creates no
+  source receipt table. Local SQLite is permitted for one process;
+  networked/multi-Runner use requires authoritative shared Postgres
+  `runtime_store` and rejects mirror mode.
+
+Existing configs without `receipts` retain the compatibility source-receipt
+behavior. New onboarding asks explicitly. Runner-ledger UPDATE requires exact
+version advancement; INSERT requires source-enforced deduplication; DELETE
+requires an exact version guard. Ambiguous post-commit outcomes stop at
+`reconciliation_required`. See
+[Guarded Single-Row CRUD Writeback](guarded-crud-writeback.md).
+
 ## Trusted context
 
 ```json
@@ -227,6 +261,39 @@ fields are documented in the JSON Schema and
 lookup, visible fields, evidence, patch allowlist, bounds/transitions, conflict
 guard, approval, and writeback mode.
 
+## Bounded set operation fields
+
+A set operation is portable reviewed authority, so it can arrive from a
+contract or the equivalent embedded capability:
+
+```json
+{
+  "operation": {
+    "kind": "update",
+    "cardinality": "set",
+    "selection": {
+      "all": [{ "column": "status", "operator": "eq", "value": "overdue" }]
+    },
+    "max_rows": 10,
+    "aggregate_bounds": [
+      { "column": "balance_cents", "measure": "before", "maximum": 50000 }
+    ],
+    "version_advance": { "column": "version", "strategy": "integer_increment" }
+  },
+  "approval": { "mode": "human", "required_role": "billing_reviewer" },
+  "writeback": { "mode": "direct_sql" }
+}
+```
+
+Set UPDATE/DELETE requires one through eight fixed equality terms in
+`selection.all`; the model cannot add or override them. Every set requires
+`max_rows` from 1 through 100 and one through eight aggregate bounds. Set
+UPDATE requires integer version advancement. Batch INSERT instead declares
+`operation.batch.items_from_arg` pointing to a bounded `object_array` argument
+and item-field dedup components. All forms require human/operator approval and
+direct SQL writeback in 1.3. See [Bounded Set
+Writeback](bounded-set-writeback.md).
+
 ## Executors
 
 ```json
@@ -247,7 +314,10 @@ guard, approval, and writeback mode.
 }
 ```
 
-Types are `sql_update`, `http_handler`, and `command_handler`. HTTP handlers use
+Types are `sql_update`, `http_handler`, and `command_handler`. The legacy
+`sql_update` executor identifier now dispatches the reviewed native operation
+(`insert`, `update`, or `delete`) and remains named for config compatibility.
+HTTP handlers use
 `url_env`, optional `POST|PUT|PATCH`, bearer auth, signing secret, and timeout.
 Command handlers use `command_env` and timeout. Secrets stay in the environment.
 See [Writeback Executors](writeback-executors.md).
@@ -382,9 +452,10 @@ capability permissions, and trusted session metadata.
 
 ## Direct SQL readiness
 
-An administrator must apply `writeback migration` once. The steady-state writer
-needs target-table SELECT/allowed UPDATE plus receipt-table
-SELECT/INSERT/UPDATE; it does not need schema CREATE.
+For `source_db` + `precreated`, an administrator applies `writeback migration`
+once. The steady-state writer needs target-table `SELECT` and only the reviewed
+operation DML plus receipt-table `SELECT`/`INSERT`/`UPDATE`; it does not need
+schema `CREATE`.
 
 ```bash
 synapsor-runner writeback migration --engine postgres --schema synapsor
@@ -393,5 +464,8 @@ synapsor-runner writeback grants --engine postgres \
 synapsor-runner doctor --config ./synapsor.runner.json --check-writeback
 ```
 
-The doctor probe uses a rolled-back receipt insert and target-table check. It
-does not mutate business rows.
+`source_db` + `auto_migrate` runs that fixed migration idempotently and needs
+bounded `CREATE`. `runner_ledger` skips source receipt DDL/DML and validates its
+authoritative ledger topology. Doctor uses rollback-only receipt and target
+checks and does not mutate business rows. See
+[Guarded Single-Row CRUD Writeback](guarded-crud-writeback.md).

package/docs/running-a-runner-fleet.md

@@ -89,7 +89,8 @@ worker termination during an open write transaction rolls back and recovers
 worker termination after commit recovers as already-applied
 source-down, read-only-ledger, and timeout readiness failures recover without restart
 shared dead letters requeue or discard with receipts and events preserved
-Postgres and MySQL pool queues fail fast at configured bounds
+Postgres and MySQL pool queues fail fast at configured bounds with retry hints
+shared runtime-store batch apply preserves one authoritative bridge and durable receipts
 backup digest and clean restore match
 retention archives before delete and preserves active proposals
 ```

package/docs/security-boundary.md

@@ -20,6 +20,13 @@ trusted context, evidence handles, query audit, local inspection, and
 proposal-first writes. Proposal workflows add full replay across evidence,
 approval, writeback receipts, and events.
 
+Prompt instructions and application validation are separate layers. Prompts
+can guide behavior but cannot supply authorization. A fixed, parameterized
+application tool can be a sound boundary for a small system; Runner adds a
+portable reviewed contract and a consistent proposal, approval, receipt,
+replay, and compensation lifecycle. See [Why Synapsor Over Prompt And
+Application Guardrails](why-synapsor-vs-app-guardrails.md).
+
 The model-facing MCP server exposes reviewed semantic tools such as
 `billing.inspect_invoice` and `billing.propose_late_fee_waiver`.
 
@@ -67,17 +74,35 @@ outside the model-facing MCP server and verifies:
 - allowed mutable columns;
 - conflict/version guard;
 - idempotency key;
+- operation-specific version or source-unique deduplication guard;
 - job expiry;
-- exactly one affected row.
+- exactly one reviewed row or every member of one bounded frozen set.
 
 If any authority check cannot be verified, the write fails closed.
 
 For direct SQL writeback, the writer connection is the env var named by the
-source `write_url_env` in `synapsor.runner.json`. Direct SQL writeback writes an
-administrator-created `synapsor_writeback_receipts` table for idempotency and
-replay. The writer needs `SELECT`/`INSERT`/`UPDATE` on that table, but not schema
-`CREATE`. If your database policy forbids the receipt table, use an app-owned
-`http_handler` or `command_handler` executor instead.
+source `write_url_env` in `synapsor.runner.json`. With `source_db` receipt
+authority, the source mutation and receipt commit atomically; the receipt table
+can be administrator-precreated or explicitly auto-migrated. With
+`runner_ledger` authority, Runner creates no source receipt table, but a crash
+after source commit and before ledger completion can require verified operator
+reconciliation. It is not distributed exactly-once. See
+[Guarded Single-Row CRUD Writeback](guarded-crud-writeback.md).
+
+Bounded-set direct writeback additionally requires a contract-fixed predicate
+or complete reviewed item list, `MAX ROWS` plus aggregate bounds,
+human/operator approval, exact frozen identities and versions, deterministic
+locks, one atomic source transaction, and per-member receipt digests. The
+model cannot supply the predicate or cap. See [Bounded Set
+Writeback](bounded-set-writeback.md).
+
+When a direct SQL capability opts into reviewed reversibility, an applied
+receipt may carry a bounded inverse containing only trusted identity, version,
+and reviewed writable values. `revert` is operator-only and creates a new
+proposal; it never mutates or approves. Compensation rechecks the exact forward
+result under the trusted tenant and uses the normal approval, receipt,
+reconciliation, and replay boundaries. See [Reviewed Reversible Change
+Sets](reversible-change-sets.md).
 
 When a capability uses an `http_handler` or `command_handler` executor, the
 same approval boundary applies. The runner sends a structured proposal/job
@@ -96,6 +121,8 @@ approve/reject actions. It does not expose raw SQL, database URLs, write
 credentials, approval tools, commit tools, or controls that widen reviewed
 tables/columns.
 
-Synapsor Runner supports reviewed single-row business actions only in the
-current alpha. It does not support arbitrary SQL, DDL, `INSERT`, `DELETE`,
-`UPSERT`, or multi-row updates.
+Synapsor Runner supports reviewed single-row CRUD and the narrowly bounded set
+path documented above. It does not support arbitrary SQL, DDL, UPSERT,
+model-generated predicates, unbounded sets, or cross-table direct writeback.
+Hard DELETE fails closed when cascades, write triggers, or required metadata
+visibility prevent a bounded-effect proof.