messagefoundry 0.2.11__tar.gz → 0.2.12__tar.gz

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/.github/workflows/ci.yml

@@ -431,6 +431,33 @@ jobs:
           MEFOR_ALLOW_INSECURE_TLS: "1"
         run: pytest tests/test_load_failover_sqlserver.py -v
 
+      - name: Run the throughput-lever backend invariants on real SQL Server (B1 inline, B2 batch-claim, ...)
+        env:
+          MEFOR_TEST_SQLSERVER: "1"
+          MEFOR_STORE_BACKEND: sqlserver
+          MEFOR_STORE_SERVER: localhost
+          MEFOR_STORE_PORT: "1433"
+          MEFOR_STORE_DATABASE: MessageFoundry
+          MEFOR_STORE_AUTH: sql
+          MEFOR_STORE_USERNAME: sa
+          MEFOR_STORE_PASSWORD: "Str0ng_P@ssw0rd!"
+          MEFOR_STORE_TRUST_SERVER_CERTIFICATE: "true"
+          MEFOR_ALLOW_INSECURE_TLS: "1"
+        # The staged-pipeline throughput levers (ADR 0057 inline fast-path, ADR 0058 batch-claim, ...) carry
+        # reliability-core invariant tests — crash-replay, poison-bound, and the per-lane-FIFO locked-head-
+        # BLOCKS (#285) gate — that are MEFOR_TEST_SQLSERVER-gated but live in their OWN files, so the named
+        # suites above never picked them up and they ran only on SQLite. Run them here against the real
+        # backend. APPEND each new lever's test file per docs/throughput-build-plan.md.
+        run: >-
+          pytest -v
+          tests/test_inline_fast_path.py
+          tests/test_batch_claim_fifo.py
+          tests/test_batch_claim_worker.py
+          tests/test_batch_claim_locking.py
+          tests/test_seq_only_fifo.py
+          tests/test_fifo_index_migration.py
+          tests/test_per_lane_wake.py
+
   # Postgres store backend (Track B): run the gated store suite against a real PostgreSQL service
   # container (Linux, so 1x minutes). Skipped on PRs like the other server-DB job to save spend; runs
   # on push to main and on-demand (workflow_dispatch — use `gh workflow run ci.yml --ref <branch>` to
@@ -504,6 +531,53 @@ jobs:
           MEFOR_ALLOW_INSECURE_TLS: "1"
         run: pytest tests/test_load_failover_postgres.py -v
 
+      - name: Run the throughput-lever backend invariants on real Postgres (B1 inline, B2 batch-claim, ...)
+        env:
+          MEFOR_TEST_POSTGRES: "1"
+          MEFOR_STORE_BACKEND: postgres
+          MEFOR_STORE_SERVER: localhost
+          MEFOR_STORE_PORT: "5432"
+          MEFOR_STORE_DATABASE: messagefoundry
+          MEFOR_STORE_USERNAME: postgres
+          MEFOR_STORE_PASSWORD: mefor
+          MEFOR_STORE_ENCRYPT: "false"
+          MEFOR_ALLOW_INSECURE_TLS: "1"
+        # The throughput levers' MEFOR_TEST_POSTGRES-gated invariants (esp. the FOR-UPDATE no-SKIP-LOCKED
+        # locked-head-BLOCKS twin of #285, and the batch FIFO/crash tests) live in their own files. APPEND
+        # each new lever's test file per docs/throughput-build-plan.md.
+        run: >-
+          pytest -v
+          tests/test_inline_fast_path.py
+          tests/test_batch_claim_fifo.py
+          tests/test_batch_claim_worker.py
+          tests/test_batch_claim_locking.py
+          tests/test_seq_only_fifo.py
+          tests/test_fifo_index_migration.py
+          tests/test_per_lane_wake.py
+
+      - name: Run the connection-scale pool-wait smoke on real Postgres (B11 wall #2, tiny pool)
+        env:
+          MEFOR_TEST_POSTGRES: "1"
+          MEFOR_STORE_BACKEND: postgres
+          MEFOR_STORE_SERVER: localhost
+          MEFOR_STORE_PORT: "5432"
+          MEFOR_STORE_DATABASE: messagefoundry
+          MEFOR_STORE_USERNAME: postgres
+          MEFOR_STORE_PASSWORD: mefor
+          MEFOR_STORE_ENCRYPT: "false"
+          MEFOR_ALLOW_INSECURE_TLS: "1"
+          # FORCE a below-default pool (4 connections) so the perf_counter-measured acquire-WAIT
+          # histogram (the PRIMARY pool-wait signal) is non-trivial even at small N: ~3N inbound workers
+          # contend for 4 connections, so acquires actually queue and the percentiles populate. The
+          # default pool (5+) would mask the wall. 4 (not 1-2) is deliberate: GET /status runs
+          # db_status()'s four sequential COUNT(*) queries on every poll, and an extremely tiny pool
+          # starves those acquires past the poller's HTTP timeout under the empty-claim herd on a slow
+          # runner — every poll then fails and the drain loops burn the test budget (the PR #675 hang).
+          # This still gives the B11 pool-wait instrumentation real regression coverage SQLite (no pool)
+          # cannot — the connscale harness OWNS the engine subprocess and inherits this env.
+          MEFOR_STORE_POOL_SIZE: "4"
+        run: pytest tests/test_connscale_postgres.py -v
+
   # Headless load test (Track B / throughput): serve the synthetic high-fan-out load config (auth
   # off, small fan-out) and drive the smoke profile through the real `python -m harness --load` CLI,
   # asserting zero message loss + all SLOs (exit 0) and uploading the JSON/CSV report. Skipped on PRs

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/.github/workflows/release.yml

@@ -347,7 +347,11 @@ jobs:
           MEFOR_FROZEN_CONSOLE_DIR: dist/messagefoundry-console
           QT_QPA_PLATFORM: offscreen
         run: |
-          pip install pytest
+          # pytest-timeout is required: the repo's pyproject `addopts` sets `--timeout=60
+          # --timeout-method=thread`, so a bare `pip install pytest` aborts the run with
+          # "unrecognized arguments: --timeout" before the smoke ever executes (it leaves the per-test
+          # watchdog intact instead of stripping addopts).
+          pip install pytest pytest-timeout
           python -m pytest tests/test_frozen_launch_smoke.py -v -p no:cacheprovider
 
       - name: Install Inno Setup (pinned)

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/.gitignore

@@ -80,6 +80,10 @@ TRANSCRIPTS.md
 # Still usable by name (they sit in PROFILES_DIR) and copy-on to the test box.
 /harness/load/profiles/hospital-baseline.toml
 /harness/load/profiles/soak-12h.toml
+# Deep-research survey of alternative OSS DB backends for the shared-DB commit-wall — local working
+# note, not a committed artifact (the conclusion is carried in the throughput ADRs / AI memory).
+/docs/research/db-commit-wall-backend-survey.md
+
 # --- MessageFoundry ---
 .mefor/
 *.db*

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/CHANGELOG.md

@@ -6,6 +6,63 @@ All notable changes to MessageFoundry are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.2.12] — 2026-07-01 — Early Access
+
+The **throughput & connection-scale wave.** The staged-queue per-message commit chain is shortened
+(opt-in inline fast-path + batch-claim, plus a result-preserving seq-only FIFO ordering that drops a
+per-handoff round-trip); a connection-scale measurement harness + read-only engine instrumentation lands;
+**per-lane wake events** (opt-in) eliminate the thundering-herd empty-claim storm that dominates at high
+connection counts; and ADR 0059's seq-only FIFO index re-key now reaches **upgraded** databases via a
+one-time on-open migration. All new *runtime* behavior is opt-in / off-by-default unless noted — the
+seq-only ordering (B3) and the index migration (B10) are result-preserving.
+
+### Added
+- **Inline Step-A fast-path** ([ADR 0057](docs/adr/0057-inline-step-a-fast-path.md)) — **opt-in per
+  inbound via `inline`**: for the pure all-deliver message (no filter/state/pass-through), fuse
+  route+transform+handoff into **one committed transaction**, cutting the per-message commit depth from 7
+  to 5 durable round-trips. Off by default → byte-identical to the split path; ineligible messages fall
+  back automatically.
+- **Batch-claim** (#671, [ADR 0058](docs/adr/0058-batch-claim-fifo-prefix.md)) — **opt-in via
+  `[store].fifo_claim_batch`** (>1): the INGRESS/ROUTED FIFO claim takes the contiguous due head-prefix in
+  one commit instead of one row per commit, processed in strict FIFO order. Default `1` = off
+  (byte-identical); preserves per-lane FIFO (#285) and at-least-once.
+- **Per-lane wake events** (#678, [ADR 0061](docs/adr/0061-per-lane-wake-events.md)) — **opt-in via
+  `[pipeline].per_lane_wake`**: a committed message wakes **only its own `(stage, lane)` worker** instead
+  of every worker of that stage, eliminating the thundering-herd empty-claim storm at high **connection**
+  counts (~1,500 inbounds). Default off + byte-identical; the FIFO claim and the lost-wakeup poll backstop
+  are unchanged (a missed wake self-heals). Env override `MEFOR_PIPELINE_PER_LANE_WAKE` for the harness A/B.
+- **Connection-scale measurement harness + read-only engine instrumentation** (#675) — a headless harness
+  that spins N inbound connections at a low per-connection rate and reads the connection-scale walls
+  (executor saturation, server-store pool wait, idle-poll storm, FD/socket count, config-reload + ACK
+  latency) vs connection count. The supporting engine instrumentation is **additive + read-only**, surfaced
+  via `/stats` + `/status`: empty-claim counters split into idle-poll vs per-commit wake-fanout, and (on a
+  server store) connection-pool acquire-wait percentiles + size/idle occupancy. Counters default to 0 /
+  `None` — byte-identical when unused.
+
+### Changed
+- **Seq-only per-lane FIFO ordering** (#673, [ADR 0059](docs/adr/0059-seq-only-fifo-ordering.md)) — the
+  per-lane FIFO claim now orders by the DB-assigned `seq` (rowid on SQLite) **alone** instead of
+  `(created_at, seq)`, and the per-insert `SELECT MAX(created_at)` clamp is removed from **every stage
+  handoff** (one fewer round-trip per produced row). **Result-preserving** (proven order-isomorphic to the
+  prior clamped ordering) and strictly more robust under clock skew / failover (`seq` has no wall-clock
+  dependence). `created_at` stays a real ingest-time/metrics timestamp — it is simply no longer an ordering
+  key. The FIFO covering indexes re-key to trail in `seq` (see the migration below).
+- **Rename-based FIFO covering-index migration** (#676, [ADR 0060](docs/adr/0060-rename-based-fifo-index-migration.md)) —
+  ADR 0059 re-keyed the per-lane FIFO indexes to trail in `seq` for the seq-only claim, but kept their names
+  under `IF NOT EXISTS` guards, so **only fresh databases** adopted the new index — an upgraded DB silently
+  kept its old `created_at`-trailing index and never got ADR 0059's throughput win. The seq-trailing indexes
+  are now named `ix_queue_fifo_in_seq` / `ix_queue_fifo_out_seq`, and a one-time, idempotent **on-open
+  migration drops the old-named index and builds the new one** on all three backends, so upgraded databases
+  adopt it. Correctness is unchanged (the claim orders by `seq`/`rowid` and names no index, so the migration
+  only restores speed). Operational notes: the first open after upgrade pays a **one-time index rebuild** on
+  the `queue` table (SQLite/Postgres blocking, SQL Server offline — bounded by live queue depth, at cold start
+  before serving); on a very large SQLite queue a *concurrent* second opener may hit a transient, non-corrupting
+  open failure during the rebuild; the shared-DB backends (SQL Server / Postgres) should upgrade **stop-the-world
+  / under a drain window** (a mixed-version fleet or a live rejoin can re-create or contend on the index); a
+  downgrade re-creates the old-named index (drop `ix_queue_fifo_in/out` manually if downgrading permanently).
+- **`/status` DB observability** — the SQLite journal mode and `synchronous` durability setting are now
+  surfaced in the DB status (`synchronous=NORMAL` remains the crash-safe-under-WAL default).
+
 ## [0.2.11] — 2026-06-29 — Early Access
 
 The **Plan-6 disaster-recovery + cloud/HA wave** — turnkey DR backup/restore-verify and a third-tier DR

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: messagefoundry
-Version: 0.2.11
+Version: 0.2.12
 Summary: Open-source healthcare integration engine — route, transform, and validate messages across many formats and connection types
 Project-URL: Homepage, https://messagefoundry.org/
 Project-URL: Documentation, https://messagefoundry.org/

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/docs/BACKLOG.md

@@ -114,6 +114,7 @@ protects the minimal-dep, on-prem, code-first identity.
 | **ASVS 4.1.5 per-message signing** | Med | **done** | — | #378 merged (`9c00b88`). Prune the stale `asvs-415-msg-signing` worktree. |
 | **#74 host CPU/mem metrics** | Med | **do-next** | S | Promoted from the #52 gap synthesis (2026-06-28) — the one zero-identity-tension additive win: host CPU/mem (psutil) on the metrics surface beside the #50 disk meter. Adds `psutil` (vet + re-lock). SQL-internals sub-scope demand-gated. |
 | **Publish the VS Code extension (Marketplace + Open VSX)** | Med | do-next | M | Publish the `ide/` extension to the **VS Code Marketplace** + **Open VSX** (`.vsix` via `vsce`/`ovsx`) so users install it instead of F5-from-source; add a CI publish leg + publisher accounts. **Owner: do soon, AFTER the planned IDE-focused improvements land** (not now — recorded 2026-06-26). Not a PyPI artifact (different ecosystem). |
+| **#95 engine-brokered AI assist** | Low | on-trigger | L | BYO already covers "use our existing AI subscription" (Copilot/Claude via `vscode.lm`, or an in-house model surfaced through VS Code) with **zero** engine work; this is the documented P1/P2 broker — engine-centralized, **per-use-audited** egress to a customer's managed subscription or self-hosted/in-house endpoint, wiring in the reserved `[ai]` `endpoint`/`provider`/`model` keys. ADR-first. **Trigger:** a customer wants engine-brokered AI to their own instance, or we have bandwidth. |
 
 **Top strategic calls** *(2026-06-19; updated 2026-06-28)*:
 1. ✅ **v0.2 locked and shipped** through `0.2.10` (#28/#29 evidence published; #22b shipped). The release-close move is done.
@@ -3324,6 +3325,240 @@ deep-dive + console-medium evaluation (2026-06-29).
 
 ---
 
+## 93. Engine + database performance monitoring — engine-wide volume/connection KPI roll-up + a throughput-overload (saturation) alert (P2)
+
+**Type:** feature — observability + alerting. A **connective** item: most of the operator-facing monitoring
+surface this asks for is **already tracked** (and partly shipped) under sibling items — this entry exists to name
+the two genuine **net-new** slivers none of them owns and to cross-link the rest, not to re-pitch built work.
+
+**Already tracked / shipped (don't duplicate).** The request — "display total message volume + connection count;
+monitor everything that affects throughput; alert when the system is becoming overloaded" — is largely covered:
+- **#21 (DONE, PR #407)** — the Prometheus `/metrics` exporter (+ optional OpenTelemetry): per-connection
+  received / delivered / errored / `queue_depth` counters + a `delivery_latency_seconds` histogram (p50/p95/p99).
+  A scraping team gets per-connection throughput/latency and can `rate()`/`sum()` it in Grafana today.
+- **#56 (SHIPPED 0.2.10, ADR 0044)** — resolvable alert-state: the `alert_instance` table, `GET /alerts/active` +
+  ack/resolve, and the real `ConnectionRow.alerts_active` count (no longer the stubbed `0`).
+- **#74 (do-next)** — host CPU / memory via `psutil` on the metrics surface.
+- **#75 (scheduled)** — the zero-install **browser ops dashboard**: live per-connection **In/Out msgs/sec** over
+  `/ws/stats`, Queued / Errors / Last-Activity. The natural home for a live throughput view.
+- **#76 (demand-gate)** — historical-metrics charting + a status-colored data-flow graph.
+- **#81 (demand-gate)** — alert escalation tiers + day/time thresholds + content (Action-Point) alerting on top of #56.
+- **#64 (measure-gated)** — the throughput-*performance* roadmap (group-commit, DB durable-write IOPS as the
+  leading driver). That item makes the engine *faster*; this item *warns* when load approaches capacity — they pair.
+- **#50 (P3)** — app-log disk metering + a message-stall rule. **#28/#29** — the load/throughput runs that set the baseline.
+
+**Net-new gap (what no sibling owns):**
+1. **An engine-wide aggregate KPI headline.** Every count above is **per-connection** (#21) or a live per-connection
+   rate (#75); nothing rolls them up into the operator's literal ask — a single **total messages through the engine**
+   figure, a **combined inbound + outbound connection count** (with running / stopped breakdown), and an **engine-wide
+   msg/s rate** — surfaced as first-class top-line KPIs on `/status` (or a sibling route), the console Engine Status
+   page, and the #75 dashboard. Reuse the existing `recent_done` rate window that already powers `backlog_seconds`
+   — don't add a second sampler. Small; mostly rides #75.
+2. **A throughput-overload / saturation alert.** Every shipped alert (the **#5** framework, **#56** state, **#81**
+   escalation) keys on an **absolute** per-connection/per-resource snapshot — depth/oldest-age ceilings — so a bursty-
+   but-draining lane and a genuinely-overloaded engine look identical until the ceiling trips. Nothing fires on the
+   *derivative*: a **rising** `backlog_seconds`, a **growing** `in_pipeline`, or **ingest rate exceeding drain rate**
+   over a sampling window = "the system is *becoming* overloaded." Add a new ADR 0014 alert event keyed on that
+   comparison (new `AlertSink` event + emit site + `AlertRule` dimension + a small per-lane / engine-wide rate
+   history), bounded by the existing `realert_seconds` throttle and routed through the same notifier/rules path. It is
+   distinct from #81 (a policy layer *on top of* existing alerts, not a new detector) and from #64 (performance tuning,
+   not operational alerting). ADR 0014 already declined timed multi-stage escalation; this adds a rate/saturation
+   **dimension**, which wants that scope decision settled first.
+
+**Database performance monitoring.** Surface the throughput-affecting **DB** signals — write/commit latency and
+connection-pool busy/wait/saturation (the pipeline is commit-bound; `[store].pool_size` exists but emits no
+saturation metric) and router/transform-worker lag — as **metrics first** (extend `/metrics` + `db_status()`), then
+optionally as overload-alert inputs. `storage_threshold` today alerts only on DB **file size** vs
+`[retention].max_db_mb`, not commit/pool health. **SQL-internal** DB metrics stay **DBA-delegated / demand-gated**
+(parity with #74's SQL-internal sub-scope and the DB-tier-HA decline).
+
+**Why P2 / on-trigger.** #21 + the scheduled #74/#75 already answer the "where's the dashboard?" ask cheaply; the
+net-new is the small aggregate KPI roll-up (rides #75) + the overload/saturation alert, which is genuine new engine
+work and matters most once a real high-volume estate exists to overload. **Trigger:** a pilot/production estate
+approaching the commit-bound capacity ceiling that needs an *early-warning* overload signal rather than after-the-fact
+`queue_buildup`; calibrating the threshold wants a #28/#29/#64 capacity baseline first. Relates to **#21**, **#56**,
+**#64**, **#74**, **#75**, **#76**, **#81**, **#50**, **#28**/**#29**, and the **#5** AlertSink/rules framework it extends.
+
+**Source:** owner request (2026-06-30) — "engine + database performance monitoring and alerting; display the total
+volume of messages going through the engine and the number of connections; monitor for all the things that affect
+throughput and alert when the system(s) are becoming overloaded." Overlap against the existing observability/alerting
+items (#21/#56/#64/#74/#75/#76/#81) reconciled the same day.
+
+---
+
+## 94. External BLOB-server offload for embedded documents — replace inline base64 with a stored-object pointer (OBX-5 RP) (P2, on-trigger)
+
+**Type:** feature — storage minimization + customer-infrastructure integration. The ingest-time **offload**
+half of **#47** (its deferred fork (b)), but targeting the **customer's existing object/BLOB store** instead
+of a MessageFoundry-internal attachment table — and replacing the inline blob with an **in-message pointer**,
+not a private reattach token.
+
+**The ask.** Large base64 embedded documents (PDF reports, CCD/C-CDA, scanned images) ride inline in **OBX-5**
+(ED data type) and generically via the ADR 0028 `mfb64:v1:` carriage marker
+([`adr/0028-base64-binary-carriage-codec.md`](adr/0028-base64-binary-carriage-codec.md)). Today they are stored
+verbatim in the raw message at **every** persisted stage (`ingress` → `routed` → `outbound`), bloating the store
+far out of proportion to message count (#47's premise). Instead of pruning them *after* a window (#47(a)) or
+carrying them more compactly *inside* our store (#62), **offload the blob to the customer's BLOB server at
+ingest, take back the storage key/URL it returns, and embed that pointer into the corresponding OBX segment** —
+so the bulky document never persists in our store at all.
+
+**Why distinct from the siblings.**
+- **#47(a)** prunes the embedded doc *after* a per-connection window — the blob still bloats all three stages
+  until the window elapses, and it stays in our store meanwhile. This eliminates it *from the start*.
+- **#47(b)** is the same ingest-time-offload shape but offloads to a **MessageFoundry-managed attachment store**
+  (Mirth's `d_ma<channelId>` table + `${ATTACH:...}` token, reattached on outbound). This offloads to
+  **infrastructure the customer already owns** and leaves a **standards-shaped pointer in the message**, not a
+  private token.
+- **#62** keeps the bytes in our store, just as `VARBINARY`/`BLOB`/`bytea` instead of base64-in-text. Here the
+  bytes **leave** our store entirely.
+
+**Design forks (for the ADR):**
+- **Pointer representation.** Replace the OBX-5 **ED** embed with the HL7 **RP (reference pointer)** data type — a
+  `<pointer>^<application ID>^<type of data>^<subtype>` reference downstream systems understand natively — versus
+  an opaque MessageFoundry token (#47(b)-style) that we must reattach before delivery. RP is interoperable but
+  assumes the partner can dereference the BLOB; a token keeps the message self-contained but makes us re-fetch +
+  re-embed on outbound. For the generic `mfb64:v1:` carriage, a sibling `mfref:`-style pointer marker. **Never
+  string-slice raw HL7** (CLAUDE.md §8) — rewrite via the parsed model/codec and re-encode.
+- **Credential-bearing pointers — embed a reference, not a capability.** The message must carry a pointer a
+  consumer can resolve, but a BLOB store often hands back (or we would mint) a **presigned URL / SAS token with the
+  access grant baked into the string**. That must **not** be what we persist: a presigned URL in OBX-5 is a bearer
+  credential to PHI living in a persisted-and-forwarded artifact (store, outbox, the partner's inbox, our logs), it
+  **expires** — colliding with at-least-once **replay**, queued **retries**, **dead-letter**, and **retention** (a
+  message re-sent past the TTL carries a dead pointer) — and it can't be revoked independently of the document.
+  Separate the two capabilities: the **upload** grant (the presigned PUT, or MessageFoundry's own write creds) is
+  used **once and discarded**; what we **embed** is a **stable, opaque, non-capability reference** — ideally a
+  content-addressed object key plus the store identity (the HL7 **RP** components map cleanly: *Application ID* =
+  which BLOB store, *Pointer* = the opaque key), with the consumer authenticating to the store with its **own**
+  credentials (it owns the store — the premise of this feature). If a partner genuinely needs a no-auth
+  dereferenceable URL, **mint a short-lived presigned URL late, at delivery** (the reattach-on-outbound fork below),
+  never at ingest and never persisted — so the capability exists only transiently on the wire within a bounded TTL.
+  Clean default: MessageFoundry writes with its own creds, embeds the opaque key, readers use theirs, and no
+  credential URL ever touches the store or the logs.
+- **Reattach-on-outbound or not.** If the receiving partner reads the BLOB itself, the pointer *is* the
+  deliverable. If it needs the actual document, MessageFoundry must **re-fetch from the BLOB and re-embed** on the
+  outbound — or **mint a fresh short-lived pointer** at send time (above) — a new read side-effect + egress
+  dependency on delivery. Per-outbound choice.
+- **Where the offload runs vs the reliability invariant.** This is a **write side-effect**, which collides with
+  the "routers/transforms must be pure, every stage is at-least-once / re-runnable" invariant (CLAUDE.md §2). A
+  stage re-run must not double-store or orphan blobs — favor **content-addressed keys** (hash of the bytes) so a
+  PUT is idempotent. And it adds an **external dependency** to the path: if the offload sits *before* the ACK
+  (alongside ingress persistence), a BLOB-server outage blocks intake/ACK; if it sits as its own pipeline stage
+  *after* the ACK, intake survives but a failed offload dead-letters post-ACK (no NAK) — the ADR must pick.
+
+**Scope (when built):**
+- A **pluggable BLOB connector** registered like the destination transports (`transports/`, registry — never
+  special-cased in `pipeline/`): S3 / Azure Blob / GCS / on-prem object store / plain HTTP PUT, selected +
+  configured per connection. Gated by `[egress].allowed_*` allow-lists; credentials via `env()` / `MEFOR_*` (the
+  connector-credential SecretProvider-seam candidate). Off the event loop.
+- A **per-connection offload setting** (size threshold + target BLOB connection), layered over a global default —
+  the same **global-default + per-connection-override** model as FIFO / `RetryPolicy` / #34 / #47, authored on the
+  inbound `ConnectionSpec` and/or `connections.toml` (ADR 0007) so it stays hand-/GUI-editable.
+- Target **both** carriage forms (HL7 OBX-5 ED and the generic `mfb64:v1:` marker) across **all three** backends
+  (SQLite / Postgres / SQL Server). Preserve every invariant — never delete the row, message stays parseable after
+  the rewrite, **one audit entry per offload** (key + size + content-type + connection, no content). Offload is
+  irreversible from our side once the inline bytes are dropped — surface a distinct flag so an operator viewing the
+  message knows the document was externalized vs never present, and audit any later **retrieval**.
+
+**PHI note + scope boundary.** Offloading *shrinks our* at-rest PHI footprint (a data-minimization win) — the
+bulky document leaves our store for the customer's BLOB. **The security of that BLOB server is explicitly out of
+scope:** PHI handling, **encryption-at-rest**, and access control on the customer's store are the **customer's**
+responsibility — the same trust posture we already take toward a customer database in `db_lookup`
+([ADR 0010](adr/0010-handler-callable-db-lookup.md)). MessageFoundry treats the BLOB server as trusted customer-owned
+infrastructure and does **not** encrypt the offloaded objects or enforce remote-store PHI controls itself. What
+stays **in** scope (our responsibility): **never log a presigned/SAS URL or an identifier-bearing object key**
+(§9) — the former is a bearer credential to PHI, the latter is PHI itself; an opaque, auth-gated reference is safe
+to log. And **audit each retrieval** as a PHI access. (Logging is still stdlib with no structlog redaction yet, so
+this is a deliberate connector-level "log the object key/length, never the signed URL" discipline, not an
+automatic scrub.) The customer's BAA must cover the BLOB store; restate this boundary in [`PHI.md`](PHI.md) when
+built.
+
+**Why P2 / on-trigger.** This is the strongest store-bloat lever for heavy document feeds (radiology PDFs, CCDs)
+and the cleanest fit for a customer who **already runs** object/BLOB infrastructure and wants their documents
+living there. But it is a side-effecting pipeline change touching the purity / at-least-once invariant **plus** a
+new connector family — it wants its own ADR (the forks above) before code, and is not an open exposure on the
+shipping config. **Trigger:** an adopter with an existing BLOB/object store and a document-heavy feed who wants the
+documents offloaded out of our store. Relates to **#47** (the in-store prune/offload sibling — shared
+per-connection plumbing; this realizes its deferred fork (b) against external storage), **#62** (in-store binary
+carriage), **#34** (per-connection retention), **ADR 0028** (base64 carriage), **ADR 0007** (`connections.toml`),
+and the connector-credential **SecretProvider** seam.
+
+**Source:** owner request (2026-06-30) — "integrate with the customer's existing BLOB servers to offload base64
+documents; eliminate the base64 documents from our data store — instead get a pointer back from the BLOB and embed
+that into the corresponding OBX segment." Reconciled against the in-store siblings #47 / #62 the same day.
+
+---
+
+## 95. Engine-brokered AI assistance — integrate the IDE coding assistant with a customer's managed AI subscription or in-house LLM instance (P3, on-trigger)
+
+**Type:** feature — AI governance + customer-infrastructure integration. Turns the **reserved-but-unused**
+`[ai]` broker config keys into a real integration: let the **engine broker** the IDE assistant's model calls to a
+provider the *customer already runs* — their own cloud AI subscription (Azure OpenAI, Anthropic/Bedrock, an internal
+Copilot-compatible gateway) or a **self-hosted / on-prem LLM endpoint** (vLLM, Ollama, an internal inference service)
+— under central, **per-use-auditable** egress control. The policy model, config schema, RBAC, and policy endpoint
+**already exist** ([`AI.md`](AI.md)); this builds the broker they were designed for.
+
+**Already there (don't duplicate).** A customer's existing AI **subscription** is *already* the integration point
+today, via **BYO** ([`../ide/src/chat.ts`](../ide/src/chat.ts)): the assistant is provider-agnostic and uses whatever
+model the developer picked in VS Code's Chat view (Copilot / Copilot Enterprise under the org BAA, Claude, etc.)
+through the `vscode.lm` Language Model API — and **any in-house instance that registers as a VS Code language-model
+provider** (a Copilot-compatible internal proxy or a custom chat-model extension) is picked up the same way,
+engine-blind. The governance around it is built too — the `[ai]` policy (`mode` × `data_scope`,
+production-posture-clamped), the `ai:assist` RBAC permission, `GET /ai/policy` + the `messagefoundry ai-policy` CLI,
+and the central-*off* switch honored on every workstation. What is **not** built is the **engine-brokered** path:
+`managed_claude` / `managed_claude_baa` are accepted as policy values but the IDE deliberately refuses to service them
+(it will **not** silently fall back to BYO), and the `provider` / `model` / `baa_attested` / `endpoint` config keys
+are **accepted but unused** — placeholders the broker was meant to consume.
+
+**Net-new gap (what no sibling owns):**
+1. **The engine-side broker.** AI.md's *Future direction* (P1/P2) puts model egress behind the **engine** — not the
+   dev's IDE — so a central operator controls and **per-use audits** every call, and `phi` scope becomes reachable
+   only under `managed_claude_baa` over a **BAA + zero-data-retention** connection. None of this exists: it needs a
+   new engine API surface (the engine proxies the chat request), the IDE client switching from `vscode.lm` to the
+   engine for managed modes, and per-use egress auditing (today even policy *reads* aren't audited — that arrives
+   *with* the broker).
+2. **A generic customer-endpoint mode, beyond Anthropic-managed Claude.** The only future modes named today
+   (`managed_claude` / `managed_claude_baa`) are framed around an **Anthropic-managed** Claude. A customer's **own
+   subscription** (their Azure OpenAI / Bedrock keys, their internal gateway) or a **self-hosted endpoint** is a
+   *different* shape: the customer supplies `endpoint` + `provider` + `model` + credentials and MEFOR just brokers to
+   it. That wants either a new `managed_endpoint` (engine-brokered, customer-keyed) mode or an explicit
+   generalization of `managed_claude`, finally wiring in the reserved `endpoint` / `provider` / `model` keys.
+
+**Design forks (for the ADR):**
+- **Why broker at all when BYO already works?** BYO's limit is that it is **dev-machine-local and engine-blind** —
+  ops can centrally turn it *off* and cap scope, but cannot *see* or *audit* individual calls, and the model is
+  whatever the dev configured in VS Code. The broker buys central egress control, per-use audit, and a single
+  operator-pinned `endpoint` / `model` — at the cost of routing AI traffic through the engine. Some customers want
+  exactly the opposite (keep AI entirely off the engine), so this is **additive, never a replacement** for BYO.
+- **`managed_endpoint` vs generalize `managed_claude`.** A new mode keeps the existing Claude modes clean;
+  generalizing avoids mode-proliferation. Either way the IDE's current "managed → disabled" branch flips to
+  "managed → call the engine broker."
+- **Credentials + egress.** Customer keys / endpoint via `env()` / `MEFOR_*` (the connector-credential
+  **SecretProvider** seam), gated by an `[egress].allowed_http` allow-list like `fhir_lookup` / SMART; the broker
+  call runs **off the event loop**. A self-hosted endpoint (vLLM / Ollama) often needs no BAA (on-prem) — but the
+  **`data_scope` ceiling still applies**: `phi` stays reachable *only* under the BAA + ZDR attestation, never merely
+  because the endpoint is on-prem.
+- **PHI boundary unchanged for the MVP scopes.** Until de-id wiring into the AI scope path and the broker land
+  together, the assistant still attaches **`code_only`** context regardless of mode — the broker changes *who makes
+  the call and how it's audited*, not *what data* may be sent without a posture change.
+
+**Why P3 / on-trigger.** BYO already covers "use our existing AI subscription" for the common case
+(Copilot-under-BAA, or an in-house model surfaced through VS Code) with **zero** engine work — so this is genuine new
+engine + IDE + audit surface that earns its cost only when a customer specifically wants **engine-centralized,
+audited** AI egress to **their** managed / self-hosted endpoint (e.g. a security team that mandates all AI traffic
+flow through one audited choke point, or an estate whose only LLM is an internal one not exposed to VS Code).
+**Trigger:** a customer asks for engine-brokered AI to their own subscription / in-house instance, **or** we have the
+bandwidth to build out the documented P1/P2 broker. ADR-first (the forks above). Relates to [`AI.md`](AI.md) (the
+policy model + reserved keys this realizes), [`PHI.md`](PHI.md) §9 (de-id, the gate to scopes above `code_only`), the
+**SMART** / `fhir_lookup` egress-allow-list + off-loop precedent ([ADR 0024](adr/0024-smart-backend-services-token-provider.md) /
+[ADR 0043](adr/0043-fhir-read-lookup.md)), and the connector-credential **SecretProvider** seam.
+
+**Source:** owner request (2026-06-30) — add the engine-brokered "integrate the IDE coding assistant with a
+customer's existing AI subscriptions or in-house instances" capability as a demand-/bandwidth-gated item; build when
+a customer wants it or when we have bandwidth. The already-shipped BYO coverage + the reserved broker config keys were
+reconciled the same day.
+
+---
+
 *Everything else from the 2026-06-10 full-codebase review (1 critical, 13 high, 33 medium, 31 low —
 78 findings) has been remediated; see the review report's §6 action order. The two items it still
 sourced — **#1 (SQL Server concurrency)** and **#2 (console off-thread)** — are now both **DONE**

messagefoundry-0.2.12/docs/BRAND.md

@@ -0,0 +1,103 @@
+<!-- Source of truth: MessageFoundry Wordmark Guidelines (brand guidelines, June 2026), MEFOR-ORG. -->
+
+# MessageFoundry — Brand & Wordmark Guidelines
+
+How to set the **MessageFoundry** wordmark — the amber accent, type, the ™ mark, and reproducing it
+in Microsoft Word.
+
+## The wordmark
+
+The wordmark is the single word **MessageFoundry** — never split, re-cased, or re-colored.
+
+- **"Message"** — the base text color: white on dark, near-black **`#0f172a`** on light.
+- **"Foundry"** — **always molten amber `#f59e0b`**, in every context.
+- **Casing** — camelCase `MessageFoundry`, one word, no space.
+- **Code name is separate** — the package/import token `messagefoundry` is lowercase plain text;
+  **never** style it as the wordmark.
+
+## Color
+
+Only **"Foundry"** is colored, and only with the amber below — the same value the logo, favicon, and
+Open-Graph art use. The other tones belong to the wider palette, **not** the wordmark.
+
+**Molten amber — the "Foundry" accent:** HEX `#f59e0b` · RGB `245, 158, 11` · CSS `--accent`.
+
+**Supporting palette — _not_ for the wordmark:**
+
+| Hex | Name | Use |
+|---|---|---|
+| `#f97316` | molten orange | headline gradient only |
+| `#7c2d12` | deep accent ink | accent text on light |
+| `#0b1220` | deep slate | dark backgrounds |
+| `#0f172a` | primary text | "Message" on light |
+
+## Typography & the amber rule
+
+- **Typeface** — system sans, **Segoe UI** on Windows (Aptos or Calibri are acceptable fallbacks).
+  No serif, script, or display faces.
+- **Weight** — 800 / extra-bold (or the boldest available).
+- **Tracking** — letter-spacing −0.02em (≈ "Condensed, By 0.2 pt" in Word). Never loosen it.
+
+> **Amber is for marks & headings — never body text.** Color "Foundry" amber only in the **wordmark,
+> headings, and mastheads**. In running copy, write MessageFoundry in the **normal text color**.
+> (Applies to documents as well as the website.)
+
+## The ™ trademark mark — when to include it
+
+MessageFoundry and the MessageFoundry wordmark are trademarks of **MEFOR-ORG**. Use the ™
+**deliberately, not on every mention.**
+
+**Include the ™:**
+- On the primary logo lockup (anvil + wordmark) in a header / cover.
+- On the **first or most prominent** appearance in a document.
+- In standalone logo use — slides, letterhead, avatars.
+
+**Leave it off:**
+- On repeated inline mentions in running text (once is enough).
+- In code, URLs, file names, and the package name `messagefoundry`.
+- Where a legal trademark line already states ownership.
+
+**How it's set:** a superscript ™ right after "Foundry", about **0.6–0.7× the wordmark size**, in a
+**muted gray, medium weight**.
+
+## Clear space, size & backgrounds
+
+- **Clear space** — keep margin equal to the cap-height of the "M" on all sides (about the anvil's
+  height for the lockup).
+- **Minimum size** — not below ~9 pt in print / ~16 px on screen; keep the anvil mark ≥ ~20 px.
+- **Lockup** — anvil mark, a gap of ~half the mark's width, then the wordmark, vertically centered.
+- **Contrast** — amber on white is borderline for small text — another reason to keep the accent to
+  large/heading use, never body copy.
+
+## Recreating the wordmark in Microsoft Word
+
+No special font or asset needed — Word users already have the typeface. The one detail to get exactly
+right is the amber.
+
+1. Type it as one word — `MessageFoundry` (no space), then apply **Bold**.
+2. Set the font — **Segoe UI** (Windows default); Aptos or Calibri are acceptable fallbacks.
+3. **Color only "Foundry"** — select the letters `Foundry` → Home → Font Color ▾ → More Colors →
+   Custom → Hex `F59E0B` (RGB 245, 158, 11). Leave "Message" on the default color.
+4. Optional tracking — select the word → Ctrl+D → Advanced → Spacing **Condensed, By 0.2 pt**.
+5. Add the ™ (prominent use only) — after "Foundry" type `(TM)` (auto-corrects to ™, or Alt+0153) →
+   select it → **Superscript** (Ctrl+Shift+=), optionally **gray and one size smaller**.
+6. Full lockup — Insert → Pictures → the anvil logo → size to the wordmark's cap-height → place left
+   with a small gap → select both and **Group**.
+
+**Don't** recolor or outline the letters, stretch/skew the text, or apply WordArt, shadow, or glow.
+It's bold sans type with one amber word — nothing more.
+
+### Do / Don't
+
+| Do | Don't |
+|---|---|
+| One word, camelCase: **MessageFoundry** | Add a space, or lowercase the brand in prose |
+| Color only "Foundry", only amber `#f59e0b` | Color the whole word, or color "Message" |
+| Bold, clean system sans | Tint "Foundry" amber in body text |
+| ™ on the lockup and first prominent use | Use a different orange or the gradient on the wordmark |
+| Keep clear space and contrast | Stretch, skew, outline, or repeat ™ everywhere |
+
+---
+
+© 2026 MEFOR-ORG, a non-profit. MessageFoundry and the MessageFoundry wordmark are trademarks of
+MEFOR-ORG. The amber accent is `#f59e0b` (RGB 245, 158, 11).

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/docs/CLUSTERING.md

@@ -247,6 +247,13 @@ does not replicate the store itself.
 Like Rhapsody/Corepoint, clients reach "the engine" through a **floating VIP or load balancer**, not a
 fixed node — so a failover is transparent to senders (modulo a reconnect):
 
+> **Planned alternative — engine-managed VIP (Windows-only).** [ADR 0056](adr/0056-engine-managed-vip-failover.md)
+> proposes an **opt-in** mode where the **engine itself** owns the VIP (no external LB/VRRP/WSFC), moving it
+> in lockstep with the leadership lease. It is **Windows-only** and **not yet built**. Until it ships — and
+> on **Linux/containerized** deployments, which it does **not** cover — use the external floating VIP / LB
+> described here, which stays the **cross-platform** default and the recommended posture for the strictest
+> split-brain guarantee.
+
 - **MLLP / TCP inbound (per listener).** Use a VIP per inbound port whose health check is a **TCP
   connect to that port**. Because only the **primary** binds the port (the active-passive graph gating),
   the check passes only on the primary, so the VIP routes inbound traffic to it automatically; on

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/docs/CONFIGURATION.md

@@ -54,6 +54,7 @@ backend that consumes the server-DB keys lands incrementally (the settings + val
 | `backend` | enum | `sqlite` | `sqlite` · `sqlserver` · (later `postgres`/`mysql`/`oracle`) |
 | `path` | str | `./messagefoundry.db` | SQLite only |
 | `synchronous` | enum | `normal` | SQLite: `normal`/`full` |
+| `fifo_claim_batch` | int | `1` | all backends (ADR 0058). Max rows the **INGRESS/ROUTED** FIFO claim takes per commit. `1` = **OFF** (the workers claim one row per commit — byte-identical to before). `> 1` (clamped `1..64`) claims the **contiguous due head-prefix** in one commit and then processes each row in strict FIFO order with its own off-loop route/transform + separate handoff, amortizing the standalone claim commit toward 1/N. A not-due or producer-locked head still blocks the lane (strict per-lane FIFO, #285). The **outbound/delivery** claim is never batched. Opt-in throughput tuning (recommend `8`–`16`); size against worst-case message size, since N decrypted bodies are resident per lane between the claim and the N handoffs. |
 | `encryption_key` | secret | — | **env only** (`MEFOR_STORE_ENCRYPTION_KEY`); base64 32-byte **active** key — when set, PHI columns (`raw`/`payload` + `error`/`last_error`/`detail`) are AES-256-GCM-encrypted at rest. Mint one with `messagefoundry gen-key`. Empty = off. See [PHI.md §3](PHI.md#3-encryption-at-rest). |
 | `encryption_keys_retired` | secret | — | **env only** (`MEFOR_STORE_ENCRYPTION_KEYS_RETIRED`); comma-separated base64 **decrypt-only** keys kept available during a rotation until `messagefoundry rotate-key` finishes re-encrypting under the active key (ASVS 11.2.2). |
 | `key_provider` | enum | `auto` | selects **how** the active/retired DEK bytes are *sourced* — never how they are used (the cipher, keyring, and `mfenc:v1` format are unchanged; ADR 0019, ASVS 13.3.3). `auto` (default) is the env-then-DPAPI ladder, **byte-identical** to the pre-seam behavior; `env`/`dpapi` pin a single built-in source; `aws_kms`·`azure_kv`·`gcp_kms`·`vault`·`pkcs11` envelope-decrypt a wrapped DEK inside an HSM/KMS/Vault (lazy **optional extras — not built yet**; selecting one **fails closed** at `serve`, never a silent downgrade). Names a *provider*, not key material, so it is **not** a secret. |
@@ -467,6 +468,7 @@ Because the local diff is cheap and PHI-safe it is **on by default** (zero phone
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `max_correlation_depth` | int (≥1) | 8 | **Re-ingress loop cap** (ADR 0013 Increment 2). When a captured reply is re-ingressed (`reingress_to=`/`Loopback()`), the re-ingressed message carries a `correlation_depth`; a message at this depth still routes, but the next hop (depth+1) **dead-letters** its re-ingress work-row and marks the origin `ERROR`. Coarse by design — it bounds *total work*, not topology, so a chain that legitimately bounces A→B→A a few times needs headroom. 8 is safe for typical request→response→route feeds; raise it for deep correlation chains, lower it to fence a misbehaving loop. (A value of 0 would dead-letter every re-ingress, so the floor is 1.) |
+| `per_lane_wake` | bool | `false` | **Per-lane wake events** (B12, [ADR 0061](adr/0061-per-lane-wake-events.md)). **Reliability-core, default-OFF.** When `false`, a committed message wakes every worker of its stage via an engine-wide event (the historical behavior). When `true`, it wakes **only its own (stage, lane) worker**, eliminating the thundering-herd empty-claim storm that dominates at high **connection** counts (~1,500 inbounds). Correctness is unchanged (the FIFO claim + the 0.25 s lost-wakeup poll backstop are untouched; a missed wake self-heals within the poll). **Read once at engine start — a `/config/reload` does NOT toggle it (restart to change).** Env override (for the connection-scale harness A/B): `MEFOR_PIPELINE_PER_LANE_WAKE=true`. Leave `false` until connection scale (hundreds+ of inbounds) makes the herd a measured cost. |
 
 ### `[diagnostics]`
 The Corepoint-style **event log** (#46) — a metadata-only record of connection lifecycle / pre-ingress

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/docs/adr/0001-staged-pipeline-architecture.md

@@ -134,7 +134,12 @@ expectation, not just a convention).
 Use a **single generic staged-queue table** with a `stage` discriminator, mirroring the proven outbox
 shape: `id, message_id, stage, payload, status, attempts, next_attempt_at, last_error, created_at,
 updated_at`, indexed `(stage, status, next_attempt_at)` for the claim and ordered `(created_at, rowid)`
-for FIFO. The current **outbox becomes "the outbound stage's rows"** of this model. The Phase-1 store
+for FIFO. **(Superseded by [ADR 0059](0059-seq-only-fifo-ordering.md): per-lane FIFO now orders by
+`seq` alone — SQLite `rowid`, SQL Server `BIGINT IDENTITY`, Postgres `BIGSERIAL` — dropping the
+`_fifo_created_at` write-time clamp. `created_at` stays a real ingest-time/metrics timestamp but is no
+longer the ordering key; one serial writer per lane makes seq order == receive order with zero
+wall-clock dependence. The reliability invariant and the #285 no-skip / head-of-line-blocking lock
+semantics below are unchanged.)** The current **outbox becomes "the outbound stage's rows"** of this model. The Phase-1 store
 methods generalize to take a `stage`: `claim_next_fifo` / `claim_ready`, `mark_done` / `mark_failed` /
 `dead_letter_now`, `reset_stale_inflight`, `pending_depth`. The Phase-1 worker (FIFO/unordered,
 retry-policy, internal-error policy, buildup alert) becomes a **per-stage worker** parameterized by

{messagefoundry-0.2.11 → messagefoundry-0.2.12}/docs/adr/0055-group-commit-durable-write.md

@@ -10,6 +10,14 @@
   · [0005](0005-transform-accessible-state.md) (the read-through state caches) · [0037](0037-multi-process-sharding-l3.md)
   (sharding — composes per shard) · [CLAUDE.md](../../CLAUDE.md) §2 (reliability + count-and-log invariants)
   · BACKLOG #64 · `docs/research/db-commit-wall-backend-survey.md` (gitignored)
+- **Correction (2026-06-30):** the `commit_delay`/`commit_siblings` "native group-commit" referenced
+  below is a **PostgreSQL-ONLY** GUC. **SQL Server has no durability-neutral group-commit knob** — its
+  only commit-coalescing control is `DELAYED_DURABILITY`, which *relaxes* durability (it could ACK a
+  crash-erasable message) and is therefore **rejected** for the PHI store. So the "server-DB native
+  group-commit" path here is Postgres-only; SQL Server's scale path is the concurrent pool + sharding
+  ([ADR 0037](0037-multi-process-sharding-l3.md)), not a native GUC. **As built (#660) the committer is
+  SQLite-only**; Postgres `commit_delay` adoption remains a future **gated, off-by-default** increment,
+  measurement-deferred until sharding supplies enough concurrent in-flight commits to coalesce.
 
 ---