messagefoundry 0.2.12__tar.gz → 0.2.14__tar.gz

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/.github/workflows/benchmark.yml

@@ -84,7 +84,7 @@ jobs:
       MEFOR_STORE_DATABASE: messagefoundry
       MEFOR_STORE_USERNAME: postgres
       MEFOR_STORE_PASSWORD: mefor
-      MEFOR_STORE_POOL_SIZE: "5"
+      MEFOR_STORE_POOL_SIZE: "40"  # track the shipped default (ADR 0062: 5 -> 40)
       MEFOR_STORE_ENCRYPT: "false"  # plaintext to the container; the TLS-hardening guard needs the escape
       MEFOR_ALLOW_INSECURE_TLS: "1"
     steps:
@@ -151,7 +151,7 @@ jobs:
       MEFOR_STORE_AUTH: sql
       MEFOR_STORE_USERNAME: sa
       MEFOR_STORE_PASSWORD: "Str0ng_P@ssw0rd!"
-      MEFOR_STORE_POOL_SIZE: "5"
+      MEFOR_STORE_POOL_SIZE: "40"  # track the shipped default (ADR 0062: 5 -> 40)
       MEFOR_STORE_TRUST_SERVER_CERTIFICATE: "true"
       MEFOR_ALLOW_INSECURE_TLS: "1"
     steps:

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/.github/workflows/ci.yml

@@ -569,7 +569,7 @@ jobs:
           # FORCE a below-default pool (4 connections) so the perf_counter-measured acquire-WAIT
           # histogram (the PRIMARY pool-wait signal) is non-trivial even at small N: ~3N inbound workers
           # contend for 4 connections, so acquires actually queue and the percentiles populate. The
-          # default pool (5+) would mask the wall. 4 (not 1-2) is deliberate: GET /status runs
+          # default pool (40) would mask the wall. 4 (not 1-2) is deliberate: GET /status runs
           # db_status()'s four sequential COUNT(*) queries on every poll, and an extremely tiny pool
           # starves those acquires past the poller's HTTP timeout under the empty-claim herd on a slow
           # runner — every poll then fails and the drain loops burn the test budget (the PR #675 hang).

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/CHANGELOG.md

@@ -6,6 +6,86 @@ All notable changes to MessageFoundry are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.2.14] — 2026-07-01 — Early Access
+
+**Delta security-audit remediation.** A focused security audit of the surface added since the
+2026-06-10 full review (v0.2.0 → v0.2.13) surfaced seven verified findings; this release fixes all of
+them. No new critical, no SQL injection, no auth bypass, no RCE — the most serious was an
+unauthenticated memory-exhaustion DoS in the new default HL7 parser. Each fix ships with a regression
+test. See [`docs/reviews/DELTA-REVIEW-2026-07-01.md`](docs/reviews/DELTA-REVIEW-2026-07-01.md).
+
+### Security
+- **Bounded the built-in HL7 rich-text repetition escape** (DELTA-01/02;
+  [`_builtin_hl7.py`](messagefoundry/parsing/_builtin_hl7.py)). The tolerant built-in parser (now the
+  default hot-path backend, ADR 0054) expanded `\.inN\`-style repetition escapes with no cap, so a
+  ~15-byte inbound field (`\.in2000000000\`) allocated gigabytes synchronously on the event loop
+  **before the ACK** — an unauthenticated OOM/denial-of-service. The count is now clamped
+  (`MAX_ESCAPE_REPEAT = 512`), and a malformed count no longer raises out of a field read — that had
+  severed the connection and dropped a parseable message with **no disposition**, breaking the
+  count-and-log invariant.
+- **XML-DSig `verify()` now requires an explicit trust anchor** (DELTA-03;
+  [`parsing/xml/signature.py`](messagefoundry/parsing/xml/signature.py)). Called with neither `x509_cert`
+  nor `ca_pem_file`, it previously fell back to signxml's default of trusting **any** certificate that
+  chains to the host's system CA store (origin-blind verification); it now raises `ValueError`.
+  **Behavior change** for the opt-in `[xml]` codec — a caller must pin the expected signer or a partner
+  CA. No in-repo caller relied on the old default.
+- **FhirLookup SMART token endpoint is now egress-gated** (DELTA-04;
+  [`[egress].allowed_http`](docs/CONFIGURATION.md)). A `fhir_lookup` connection composed with
+  `with_smart_backend()` POSTs a signed `client_assertion` to its `smart_token_url`; that host was not
+  checked against the egress allowlist (only the FHIR base host was), so a crafted `smart_token_url`
+  could exfiltrate the assertion to an un-allowlisted host. The lookup and outbound arms now share one
+  gate ([ADR 0043](docs/adr/0043-fhir-read-lookup.md) §D3).
+- **Support bundle no longer discloses the store host/database; its log redaction was widened**
+  (DELTA-05/07; [`support/`](messagefoundry/support/)). The offline support bundle's `status.json`
+  carried the SQL Server `host/database` verbatim — it is now reduced to the backend kind (file basename
+  only for SQLite). The bundled-log redactor previously used a fixed HL7-segment allowlist with no
+  free-text name/DOB heuristics; it now delegates to the engine redactor
+  ([`messagefoundry.redaction`](messagefoundry/redaction.py)) for parity with stored-error redaction.
+- **Inbound HTTP listener rejects ambiguous framing** (DELTA-06;
+  [`transports/http_listener.py`](messagefoundry/transports/http_listener.py)). A duplicate
+  `Content-Length`, a duplicate `Transfer-Encoding`, or the two present together are now refused with
+  `400` per RFC 7230 §3.3.3 — closing an HTTP request-smuggling / desync surface behind a fronting proxy.
+
+## [0.2.13] — 2026-07-01 — Early Access
+
+The **store connection-scale sizing** wave — right-size the server-DB connection pool to the measured
+inverted-U optimum, guard against over-provisioning, and guarantee the message store stays unified. All
+changes are **server-DB-only**; the single-node SQLite default is unaffected.
+
+### Added
+- **Soft store-pool over-provisioning warning** ([ADR 0062](docs/adr/0062-default-store-pool-size.md)) — a
+  server-DB engine now logs an advisory `WARNING` at graph start if `[store].pool_size` is sized past the
+  connection-pool inverted-U optimum: at/beyond the ~80 catastrophic cliff, or oversized for the engine's
+  inbound-interface count (`~2.5 ×` interfaces). Advisory only — it never blocks startup; SQLite has no pool
+  so it is skipped, and the default (40) never trips it. Guards the "set a huge pool for 1500 connections"
+  footgun (which is a *sharding* problem, not a pool one).
+
+### Changed
+- **Default server-DB store connection pool size raised 5 → 40** ([`[store].pool_size`](docs/CONFIGURATION.md),
+  env `MEFOR_STORE_POOL_SIZE`; [ADR 0062](docs/adr/0062-default-store-pool-size.md)). A three-sweep
+  connection-scale study found the pool is an **inverted-U**: it helps up to ~40 per engine, and
+  **over-provisioning is catastrophic** — past ~40 the extra connections thrash one shared SQL instance
+  (WRITELOG serialization + per-message finalizer applocks), and ACK latency explodes 30–90×. 40 is the
+  measured optimum — **do not set it higher to chase connection count.** **Server-DB backends only** (Postgres
+  / SQL Server) — the default **single-node SQLite** backend is unaffected (fixed read pool + single writer;
+  never reads `pool_size`). **Existing explicit `[store].pool_size` / `MEFOR_STORE_POOL_SIZE` values are
+  unchanged** — only the unset default moves. Behavioral deltas on server-DB engines: ~**8×** the steady-state
+  DB sessions per engine, and the startup pool pre-warm rises from ~2 to **~20 connections per engine**
+  (bounded by `warm_pool_timeout`, off the intake path, self-releasing, never raises). **Connection-budget
+  caution:** `pool_size` is **per engine**, so on a shared server DB `engines × pool_size` all count against
+  one `max_connections` (Postgres default ~100 → ~2 engines at 40) — raise `max_connections`, front the DB
+  with a pooler (PgBouncer), or use SQL Server; or size `pool_size` down. **Never split the store** to fit the
+  budget ([ADR 0063](docs/adr/0063-no-split-store-unified-store-for-sharding.md)). See
+  [`docs/DEPLOY-SERVER-DB.md`](docs/DEPLOY-SERVER-DB.md) §3.
+- **No split data store: multi-shard engine sharding now requires a server DB** ([ADR 0063](docs/adr/0063-no-split-store-unified-store-for-sharding.md),
+  amends [ADR 0037](docs/adr/0037-multi-process-sharding-l3.md)). `messagefoundry supervise` with **more than
+  one shard** on a **SQLite** store is now **refused at startup** — the old SQLite-file-per-shard behavior
+  split the message store into one database per shard, fragmenting search/reporting/audit/replay. A sharded
+  deployment must share **one unified store**, so `>1` shard requires `[store].backend = 'postgres'` or
+  `'sqlserver'` (every shard connects to the same database). **A single un-sharded engine on SQLite is
+  unaffected** (byte-identical to `serve`). Migrating an existing SQLite-sharded deployment: drain each shard
+  store to empty, then re-point `supervise` at one server DB (not an offline store merge).
+
 ## [0.2.12] — 2026-07-01 — Early Access
 
 The **throughput & connection-scale wave.** The staged-queue per-message commit chain is shortened

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: messagefoundry
-Version: 0.2.12
+Version: 0.2.14
 Summary: Open-source healthcare integration engine — route, transform, and validate messages across many formats and connection types
 Project-URL: Homepage, https://messagefoundry.org/
 Project-URL: Documentation, https://messagefoundry.org/

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docker/compose.yaml

@@ -49,7 +49,7 @@ x-ha-engine-env: &ha-engine-env
   MEFOR_STORE_PORT: "5432"
   MEFOR_STORE_DATABASE: "messagefoundry"
   MEFOR_STORE_USERNAME: "mefor"
-  MEFOR_STORE_POOL_SIZE: "5"          # >= 2 required under [cluster]; >= 3 recommended
+  MEFOR_STORE_POOL_SIZE: "40"         # default (ADR 0062); >= 2 required under [cluster]
   MEFOR_CLUSTER_ENABLED: "true"
   # DEV-ONLY: the local Postgres has no CA-trusted cert, so verification can't succeed here. This
   # disables TLS verification across ALL transports — it is a CI/lab switch and must NEVER be set in

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/BACKLOG.md

@@ -426,6 +426,14 @@ client. "Moderate risk"; not verifiable in offscreen Qt tests, so it deserves a
 **FIFO per outbound connection** (simple, safe). Per-key ordering is the leading-edge refinement to
 revisit once FIFO is solid and a real workload needs the parallelism.
 
+**Naming (locked 2026-06-30):** the canonical vocabulary for this concept is **sequence key** (the
+per-message partition value — ADT = `facility + MRN`; same key → same lane → FIFO preserved,
+different keys → parallel), **sequence group** (the set of messages sharing a sequence key), and
+**sequence-keyed lanes** / **sequence sharding** (the mechanism). The older terms used elsewhere in
+this item — `partition_key` (config setting) and "order-group sharding" — refer to the same thing.
+"Order-group" was retired because "order" collides with clinical orders (CPOE/ORM); "partition key"
+was retired because "partition" is reserved for the planned store-sharding-by-channel axis.
+
 **What:** preserve order only *within* a partition key (e.g. MRN / encounter / sending facility)
 while processing *across* keys in parallel — the sweet spot between strict per-connection FIFO (safe,
 serial) and unordered parallelism (fast, unsafe). Same idiom as a broker partition key. Recent
@@ -3063,11 +3071,11 @@ parity analysis.
 
 **Cluster:** Minor gaps. **Priority:** P3. **Verdict:** demand-gate.
 
-**Scope:** Per-connection send pacing, verify the reply's MSA-2 == the sent MSH-10, and keep-alive / persistent-reconnect on MLLP/TCP outbounds.
+**Scope:** Per-connection send pacing and verify the reply's MSA-2 == the sent MSH-10. *(Keep-alive / persistent-reconnect on MLLP/TCP outbounds was promoted out to its own tracked item — see [#97](#97-keep-alive--persistent-outbound-connections--per-connector-setting-p3-on-trigger).)*
 
-**Trigger:** build when a partner needs paced sending, strict response-correlation, or a persistent connection.
+**Trigger:** build when a partner needs paced sending or strict response-correlation.
 
-**Why:** All confirmed (no pacing/keepalive; `MLLPDestination._check_ack` reads MSA-1/MSA-3 only, never matches MSA-2↔MSH-10; `mllp.py` notes persistent/pooled "can come later"). Cheap per-connection-override additions; bundle on real partner need.
+**Why:** Both confirmed (no pacing; `MLLPDestination._check_ack` reads MSA-1/MSA-3 only, never matches MSA-2↔MSH-10). Cheap per-connection-override additions; bundle on real partner need. Keep-alive is tracked separately at #97.
 
 **Source:** promoted from [#52](#52-corepoint-capability-parity-gaps--prioritized-roadmap-input-2026-06-27) gap synthesis (adversarially reviewed, 2026-06-28).
 
@@ -3559,6 +3567,81 @@ reconciled the same day.
 
 ---
 
+## 96. Built-in "setup tester" — self-service capacity estimator that benchmarks the deployed setup and reports how much traffic it can handle (P2, adopter-facing)
+
+**Type:** feature — an operator/adopter-facing **capacity self-test** shipped *with the engine*. It runs the
+same style of measurement we do for throughput testing, but as a first-class, on-demand command an adopter
+points at **their own** setup (this box, this store backend, this config) to get back an **estimate of how
+much traffic that setup can sustain**.
+
+**What:** a `messagefoundry` subcommand (e.g. `messagefoundry capacity` / `setup-test`) that drives a
+controlled synthetic load through the real engine and reports an **estimated sustainable throughput** — a
+headline **msg/s** and **msg/day** figure, ideally **per-inbound-interface** *and* engine-wide, plus the
+**limiting factor** (commit-bound / pool-saturated / CPU / disk) and a confidence caveat. It ramps to the
+saturation knee (where `in_pipeline`/`backlog_seconds` start rising faster than drain — the #93 signal) and
+reports the last rate that drained cleanly with no loss, rather than a raw peak. Reuses the **BUILT load
+harness** measurement machinery ([`harness/load/`](../harness/load/), [`docs/LOAD-TESTING.md`](LOAD-TESTING.md))
+— the rate governor, the fast correlation sink (true end-to-end latency), the drain gauge (`in_pipeline`), and
+the no-loss reconciliation — packaged as a supported engine capability rather than a dev-only tool.
+
+**Distinct from what already exists (don't duplicate):**
+- **#28 / #29 (DONE)** are the *developer/benchmark* runs of the harness against a synthetic high-fan-out
+  system-under-test, producing the **project** [`benchmarks/TUNING-BASELINE.md`](benchmarks/TUNING-BASELINE.md)
+  baseline. This item is the *adopter-run* inverse: point it at **the real deployed config on the real box** and
+  get a sizing number for *that* deployment — not a project baseline, and not something that needs the harness's
+  synthetic SUT config or the denylist-guarded estate profiles.
+- **#93 (P2)** is the *passive, runtime* counterpart — it watches real traffic and **warns** when live load is
+  approaching capacity. This item is the *active, pre-cutover* counterpart — it **measures** where that capacity
+  is in the first place, so #93's overload threshold can be calibrated against it. They pair.
+- **#40** is the enterprise-hardware CI leg; this tester is what an adopter would run **on their own hardware**
+  to reproduce a sizing number without CI access.
+
+**Design constraints (for the eventual ADR):**
+- **Must not pollute production.** A capacity run generates real store writes and would otherwise inflate the
+  true inbound counts (the count-and-log invariant persists *every* received message). It must run against an
+  **isolated/ephemeral store** (temp DB) or a clearly-marked test namespace, and never leave synthetic rows in,
+  or skew the metrics of, the live message store.
+- **Synthetic payloads only — never real PHI.** Drive it from the conformant generators
+  ([`generators/`](../messagefoundry/generators/)) / the anon framework (ADR 0030), consistent with the
+  dryrun/generate PHI rule (never against real PHI, never redirected to a committed file/CI log).
+- **Respect the per-interface bound.** Report capacity **per inbound interface** and note that a single strictly-
+  ordered feed is core-bound (owner principle: fan out feeds at source, not infinite single-feed speed) — an
+  engine-wide total is the sum across interfaces, not a single-feed number. Sequence-keyed lanes (#3) are the
+  sanctioned single-feed escape hatch when one feed outgrows a core.
+- **Name the limiting factor**, reusing the #93/#64 signals (commit/write latency, `[store].pool_size`
+  busy/wait, CPU/mem via #74, `in_pipeline` growth) so the output is *"~N msg/s, commit-bound"* rather than a
+  bare number — the sizing memos already conclude the pipeline is serial-per-inbound commit-bound, not
+  disk/CPU/pool-bound on the tested boxes.
+
+**Why P2 / on-trigger.** Turns capacity sizing — today a manual "run the dev harness + read TUNING-BASELINE by
+hand" exercise — into a **supported operation** an adopter can self-serve before a cutover (*"will this box carry
+our ~1.6M ADT/day?"*). The measurement machinery already exists; the net-new is the operator-facing command, the
+isolated-store harness, the ramp-to-knee estimator, and the capacity report. **Trigger:** a pilot/adopter needing
+a self-service pre-cutover capacity check on their own hardware (the ADR 0017 consumer-deployment pattern), or the
+#93 overload-alert threshold needing a per-deployment capacity baseline to calibrate against. Relates to
+**#28**/**#29** (the harness it wraps), **#40** (enterprise-box runs), **#64** (the throughput-performance
+roadmap), **#93** (the runtime overload-alert counterpart), and the recorded sizing work (throughput matrix /
+per-interface bound / commit-bottleneck analyses).
+
+**Source:** owner request (2026-06-30) — "add a setup tester to the engine … do tests like we're doing for
+throughput testing and report back an estimate of how much traffic the setup can handle."
+
+---
+
+## 97. Keep-alive / persistent outbound connections — per-connector setting (P3, on-trigger)
+
+> **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). Promotes the keep-alive sub-scope of [#82](#82-sender-transport-polish-bundle--pacing--msa-2msh-10-matching--tcp-keep-alive) into its own tracked item at the owner's request.
+
+**Type:** feature — a per-outbound-connection option to **hold the TCP link open across deliveries** (keep-alive / persistent) instead of the current connect-per-message behavior.
+
+**What:** an opt-in **per-connector setting** (e.g. `keepalive = true` / a `connection_mode = "persistent" | "on_demand"` knob in the outbound's `settings`, default `on_demand` so existing configs stay byte-identical) on the MLLP / raw-TCP / X12 outbound connectors. When enabled, the delivery worker reuses one open connection (reconnecting on drop/idle), rather than opening + closing a fresh socket every message as it does today. Wants: a bounded idle-close / max-lifetime, reconnect-with-backoff on a dropped link, and clean teardown on `stop()`/reload — all per-connection, with the setting validated at build (dry-run / `check`), consistent with the other outbound knobs.
+
+**Why:** confirmed gap — every TCP-family outbound opens a **fresh connection per delivery** today and there is no toggle: `MLLPDestination` ([`transports/mllp.py`](../messagefoundry/transports/mllp.py), *"Phase 1 opens a fresh connection per delivery … a persistent/pooled connection can come later"*) and `TcpDestination` ([`transports/tcp.py`](../messagefoundry/transports/tcp.py), *"Opens a fresh connection per delivery … pooling can come later"*); listed as an unbuilt MLLP feature gap in [`CONNECTIONS.md`](CONNECTIONS.md) ("keep-connection-open/pooling"). Inbound listeners are already persistent (peer-driven, idle-bounded by `receive_timeout`) — this closes the outbound half. The connect-per-message default is simple and robust to flaky peers, so this is genuinely additive and stays **off by default**; the at-least-once / idempotent-receiver contract is unchanged (a reused link that drops mid-ACK still retries, same as today). **Trigger:** a partner that needs a held-open link (a persistent-session receiver, or a high-rate feed where per-message connect setup is measurable overhead). Relates to **#82** (the sender-polish bundle this splits from — pacing + MSA-2↔MSH-10 matching stay there), **#46** (connection lifecycle events would gain reconnect/retry signals), and **#65** (outbound-connector option surface).
+
+**Source:** owner request (2026-06-30) — "add keepalive feature for outbound connections, controlled by a setting per outbound connector."
+
+---
+
 *Everything else from the 2026-06-10 full-codebase review (1 critical, 13 high, 33 medium, 31 low —
 78 findings) has been remediated; see the review report's §6 action order. The two items it still
 sourced — **#1 (SQL Server concurrency)** and **#2 (console off-thread)** — are now both **DONE**

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/CLOUD-DEPLOYMENT.md

@@ -77,7 +77,7 @@ is verified against [`config/settings.py`](../messagefoundry/config/settings.py)
 | `MEFOR_STORE_DATABASE` | `[store].database` | `messagefoundry` | the database name |
 | `MEFOR_STORE_USERNAME` | `[store].username` | `mefor` | the login |
 | `MEFOR_STORE_PASSWORD` | `[store].password` | *(secret)* | **env/secret only** — never the config file |
-| `MEFOR_STORE_POOL_SIZE` | `[store].pool_size` | `5` | `>= 2` **required** under `[cluster]`, `>= 3` recommended |
+| `MEFOR_STORE_POOL_SIZE` | `[store].pool_size` | `40` | default 40, the inverted-U optimum — don't set higher ([ADR 0062](adr/0062-default-store-pool-size.md)); `>= 2` under `[cluster]`. **Budget:** `engines × pool_size` hit one managed-PG `max_connections` (~2 engines at 40/100) — raise it, add a pooler, or size down; never split the store ([DEPLOY-SERVER-DB.md](DEPLOY-SERVER-DB.md) §3) |
 | `MEFOR_CLUSTER_ENABLED` | `[cluster].enabled` | `true` | turns on the leader lease + graph gating |
 | `MEFOR_STORE_REQUIRE_ENCRYPTION` | `[store].require_encryption` | `true` | fail-closed: refuse to start keyless (PHI at rest) |
 | `MEFOR_STORE_ENCRYPTION_KEY` | `[store].encryption_key` | *(secret)* | base64 32 bytes — `messagefoundry gen-key` |

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/CLUSTERING.md

@@ -42,7 +42,7 @@ backend   = "postgres"   # or "sqlserver"
 server    = "db.internal"
 database  = "messagefoundry"
 username  = "mefor"
-pool_size = 5          # >= 2
+pool_size = 40         # default (ADR 0062); >= 2 under [cluster]
 
 [cluster]
 enabled = true

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/CONFIGURATION.md

@@ -66,11 +66,11 @@ backend that consumes the server-DB keys lands incrementally (the settings + val
 | `username` | str | — | server DBs (required when `auth = sql`) |
 | `password` | secret | — | **env only** (`MEFOR_STORE_PASSWORD`) |
 | `encrypt`, `trust_server_certificate` | bool | `true`/`false` | TLS to the DB |
-| `pool_size` | int | 5 | server DBs |
+| `pool_size` | int | 40 | server DBs — **server-DB only** (no-op on SQLite). The inverted-U optimum (raised from 5; do **not** set higher — over-provisioning is catastrophic, [ADR 0062](adr/0062-default-store-pool-size.md)). **Per engine:** `engines × pool_size` share one `max_connections` — see [`DEPLOY-SERVER-DB.md`](DEPLOY-SERVER-DB.md) §3 |
 | `connect_timeout`, `command_timeout` | int (s) | 15 / 30 | server DBs |
 | `warm_pool` | bool | `true` | server DBs — pre-open pooled connections in the background on graph start/promotion so a connection burst (the post-promotion delivery workers, or a cold start) finds them warm instead of paying cold connects (TCP+TLS+login). Best-effort, self-releasing, **no-op on SQLite**. On by default (it touches no commit/correctness seam); set `false` to opt out on a connection-constrained/licensed site. |
 | `warm_pool_timeout` | num (s) | 15 | server DBs — upper bound on the background warm-up; on expiry it logs and continues with a partially warm pool. Must be `> 0`. A **clustered** server-DB node also rejects an **explicit** value `>= [cluster].leader_fence_timeout_seconds` (a warm should finish within the leadership term that started it); the default (15 < the 20 fence) never trips this. |
-| `warm_pool_target` | int | — | server DBs — how many connections to pre-open. Unset (default) = a safe fraction of the pool (`min(pool_size-1, pool_size//2)`), so the warm never pins more than half the pool; an explicit value is clamped to `pool_size-1`. A pool of 1 is never warmed. |
+| `warm_pool_target` | int | — | server DBs — how many connections to pre-open. Unset (default) = a safe fraction of the pool (`min(pool_size-1, pool_size//2)`), so the warm never pins more than half the pool; an explicit value is clamped to `pool_size-1`. A pool of 1 is never warmed. At the default `pool_size = 40` this is `min(39, 20) = 20` pre-opened per server-DB engine at startup. |
 | `db_schema`, `application_name` | str | — / `messagefoundry` | optional (`db_schema` ⇒ env `MEFOR_STORE_DB_SCHEMA`) |
 
 > Selecting `backend = "sqlserver"` validates that `server`/`database` (and `username` when

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/DEPLOY-SERVER-DB.md

@@ -60,12 +60,44 @@ Configure `[store]` in the service settings (full reference: [`CONFIGURATION.md`
 
 ## 3. Pool sizing
 
+The default `[store].pool_size` is **40** (server-DB only; a no-op on the SQLite default, which uses a fixed
+read pool + single writer). Raised from 5 in [ADR 0062](adr/0062-default-store-pool-size.md): the
+connection-scale study found the pool is an **inverted-U** — it helps up to ~40 per engine, and
+**over-provisioning is catastrophic** (past ~40 the extra connections thrash one shared instance — WRITELOG +
+finalizer applocks — and ACK latency explodes 30–90×). 40 is the measured **optimum**, not a floor: **do not
+set it higher to chase connection count** (that path is refuted — scale is *sharding*, not a bigger pool).
+Tunable via `[store].pool_size` / `MEFOR_STORE_POOL_SIZE`; re-measure for a materially different deployment
+shape (transform cost, message size, disk, SQL-box sizing).
+
 - **Single node:** `[store].pool_size ≥ 3` recommended. Each stage handoff is a committed round-trip and
   the per-stage workers (router, transform, per-outbound delivery) run concurrently against the pool — a
   pool of 1 serializes them against intake.
 - **Clustered (active-passive):** `pool_size ≥ 2` is **required** (a cross-section validator refuses a
   smaller pool when `[cluster].enabled`), `≥ 3` recommended — a clustered node also drives the
   membership / lease-renewal maintenance loop against the pool.
+- **Connection-budget ceiling (important on a shared server DB):** `pool_size` is **per engine**. Every
+  Postgres/SQL Server engine that shares the store — multi-process shards (all connect to the *same*
+  database) and the active-passive standby's warm pool — counts against one server limit. So budget
+  **peak connections ≈ engines × `pool_size`** (+ the standby's `warm_pool_target`) and keep it well under
+  the DB server's `max_connections` (**Postgres default ~100**; SQL Server is bounded by sessions/memory).
+  At the default `pool_size = 40`, ~2 co-located engines already reach ~100 — **a config that connected at
+  5 can fail to connect at 40.** Co-locating several server-DB engines (a sharded fan-out)? **Raise
+  `max_connections`, front the DB with a connection pooler (PgBouncer), or use SQL Server** (more sessions);
+  or size `pool_size` **down**. **Do NOT give each shard its own database/server** — that is a split store,
+  which is disallowed ([ADR 0063](adr/0063-no-split-store-unified-store-for-sharding.md)); one unified store
+  scales *vertically* (faster box/disk) + *cheaper-per-message*, never by fragmenting. Also budget the startup
+  warm burst (~20 pre-opened/engine at the default; set `warm_pool_target` or `warm_pool = false` on a
+  connection-/license-constrained site).
+- **Commit-depth at high interface counts (`fifo_claim_batch`):** the pool is not the only per-engine wall —
+  above ~48 inbound interfaces an engine becomes bound by *commits per message* on the shared store (WRITELOG
+  + the per-message finalizer applock), which a bigger pool **cannot** fix. For a **high-interface-count,
+  commit-bound** server-DB engine, set **`[store].fifo_claim_batch = 8`** (range 8–16, [ADR 0058](adr/0058-batch-claim-fifo-prefix.md)):
+  the INGRESS/ROUTED FIFO claim then takes the contiguous due head-prefix in one commit instead of one-per-row,
+  amortizing the claim commit toward `1/N` on backlogged lanes. It **preserves strict per-lane FIFO (#285) +
+  at-least-once**, is **byte-identical at `1` (the default)** and on caught-up (low-backlog) lanes, and never
+  batches the outbound/delivery claim. Size it against worst-case message size (N decrypted bodies are resident
+  per lane between the claim and its N handoffs). This — not the pool — is the *raise-interfaces-per-engine*
+  lever.
 
 ---
 
@@ -173,7 +205,8 @@ update the pin in lockstep — pin the **CA**, not the leaf, to keep rotations m
 - [ ] `[store].encrypt = true` (and **not** `MEFOR_ALLOW_INSECURE_TLS`) for any PHI deployment.
 - [ ] DB CA trusted so `trust_server_certificate = false` validates — Postgres `ssl_root_cert` **or**
       machine-store import; SQL Server **machine store only** (§5). Never `TrustServerCertificate=true`.
-- [ ] `[store].pool_size ≥ 3` (≥ 2 hard-required in cluster mode).
+- [ ] `[store].pool_size` sized (default **40**, server-DB only; ≥ 2 hard-required in cluster mode) **and**
+      the connection budget checked: engines × `pool_size` (+ standby warm) well under the DB `max_connections` (§3).
 - [ ] Bootstrap login can create the schema on first open, **or** the schema is pre-created.
 - [ ] SQL Server: RCSI enabled (auto, or pre-enabled by a DBA).
 - [ ] Source store drained (`in_pipeline → 0`) before cutover — greenfield, no in-place migration.

{messagefoundry-0.2.12 → messagefoundry-0.2.14}/docs/MENTAL-MODEL.md

@@ -422,7 +422,7 @@ The built-in HA model is **active-passive failover** (the Corepoint/Rhapsody mod
 backend = "postgres"   # or "sqlserver"; SQLite cannot cluster
 server = "db.internal"
 database = "messagefoundry"
-pool_size = 5          # >= 2 (>= 3 preferred)
+pool_size = 40         # default (ADR 0062); >= 2 (>= 3 preferred)
 
 [cluster]
 enabled = true