messagefoundry 0.2.7__tar.gz → 0.2.9__tar.gz

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/.github/workflows/ci.yml

@@ -23,9 +23,11 @@ jobs:
   test:
     name: test (${{ matrix.os }}, py${{ matrix.python-version }})
     runs-on: ${{ matrix.os }}
-    # Wall-clock backstop: bound each leg so a hung test fails in minutes, not the 6h default. The
-    # pytest-timeout watchdog (60s/test) should catch a culprit first; this is the belt-and-suspenders cap
-    # if a whole process ever deadlocks below pytest.
+    # Wall-clock backstop: bound each leg so a hung test fails in minutes, not the 6h default. Three
+    # nested watchdogs catch a hang at increasing scope (#55): the pytest-timeout per-test cap (60s,
+    # pyproject addopts) + the faulthandler belt fire FIRST and name the stuck frame; the pytest STEP
+    # has its own `timeout-minutes: 13` so a process-level deadlock below pytest fails the step fast;
+    # this job cap is the outermost belt if even that is somehow out-raced.
     timeout-minutes: 15
     strategy:
       fail-fast: false
@@ -75,10 +77,27 @@ jobs:
       - name: Type-check (mypy, strict)
         run: mypy messagefoundry
 
+      # Step-level watchdog UNDER the 15-min job cap (#55): the windows-2022 leg intermittently
+      # wedges ~25% in (a Windows ProactorEventLoop listener-teardown / socket wait that the shared
+      # session event loop can't get past), emits no output for ~12 min, then the JOB cap CANCELS it
+      # — a red ✗ with no stack and no named test. Two belts make that fail FAST and NAMED instead:
+      #   * `--timeout-method=thread` (the ONLY method on Windows — SIGALRM is POSIX-only) dumps ALL
+      #     thread stacks at the per-test cap (60s, set in pyproject addopts), naming the stuck frame.
+      #   * PYTHONFAULTHANDLER=1 + `-o faulthandler_timeout=90` is the belt that fires even when the
+      #     thread-timer CANNOT interrupt a main-thread C-level wait (a Proactor overlapped op /
+      #     blocking accept / subprocess.wait): pytest's faulthandler plugin arms a separate watchdog
+      #     thread that dumps the NATIVE stack of every thread at 90s (> the 60s pytest-timeout, so the
+      #     per-test dump is attributed first; this is the last-resort stack when the thread method is
+      #     itself out-raced). NB the plugin exposes this as the `faulthandler_timeout` ini key, not a
+      #     `--faulthandler-*` CLI flag, so it is passed with `-o`; it only DUMPS (never kills), safe.
+      # The step `timeout-minutes: 13` is the outer backstop, well under the job's 15-min cap, so a
+      # hang both watchdogs somehow miss still fails the STEP (not the whole job at the silent cap).
       - name: Tests (pytest)
+        timeout-minutes: 13
         env:
           QT_QPA_PLATFORM: offscreen
-        run: pytest -q
+          PYTHONFAULTHANDLER: "1"
+        run: pytest -q -o faulthandler_timeout=90
 
   # Build + type-check the VS Code extension, and run its integration tests. The Python jobs never
   # touch ide/, so a dep bump or IDE code change that breaks the bundle or the types would otherwise
@@ -592,7 +611,14 @@ jobs:
           # install-service.ps1 -Environment threads it into the service command + resolves
           # environments/prod.toml). INFO not DEBUG: `serve` refuses DEBUG in prod (Gate #1 — DEBUG can
           # surface PHI); INFO is enough to smoke service start + MLLP.
-          .\scripts\service\install-service.ps1 -AppExe $exe -LogLevel INFO -Environment prod
+          # -LockConfigDir: the runner checks out samples/config with the default ACL, which grants
+          # BUILTIN\Users (S-1-5-32-545) write. The in-process config-source trust guard (SEC-003,
+          # wiring._assert_safe_config_source_windows) then REFUSES to load it ("writable-by-others
+          # path") and `serve` crash-loops to SERVICE_PAUSED, so /health never comes up. Locking the
+          # config dir (strip inheritance -> SYSTEM/Administrators only; the LocalSystem service still
+          # reads it) is the production-correct posture docs/SERVICE.md prescribes, and exercises the
+          # real -LockConfigDir install path rather than the MEFOR_ALLOW_INSECURE_CONFIG_SOURCE dev escape.
+          .\scripts\service\install-service.ps1 -AppExe $exe -LogLevel INFO -Environment prod -LockConfigDir
           # The shipped samples graph includes a mutual-TLS WS-* SOAP feed (IB_IMMUNIZATION_VXU) whose
           # connector LOADS its client cert at wiring time, so the engine refuses to start the graph
           # without a readable cert+key. Mint a throwaway pair for the smoke — the smoke sends only an

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/CHANGELOG.md

@@ -6,6 +6,103 @@ All notable changes to MessageFoundry are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.2.9] — 2026-06-27 — Early Access
+
+A retention + security-hardening + observability release: per-connection retention and
+embedded-document pruning windows, dual-control config reloads with startup code-attestation,
+operational-health metrics, and a fix for the intermittent Windows listener-teardown / CI hang.
+
+### Added
+- **Per-connection retention windows (ADR 0027).** Optional `messages_days` (inbound) and
+  `dead_letter_days` (outbound) on a connection, layered over the global `[retention]` window and
+  authored on the connection spec or `connections.toml` (the same override idiom as the delivery
+  knobs): `None` inherits the global window, `0` keeps forever. The `RetentionRunner` threads a
+  per-connection cutoff through the body and dead-letter purge on **all three** store backends; the
+  never-purge-an-in-flight-body guard and the single per-pass audit row (now recording the overrides)
+  are unchanged. (#34)
+- **Embedded-document pruning (ADR 0042).** Optional `prune_documents_after` (+ a size threshold)
+  per inbound connection: after the window, bulky **base64 embedded documents** — HL7 **OBX-5 ED**
+  and the generic `mfb64:v1:` carriage — are stripped **in place** to a small size/content-type
+  tombstone (via the parsed model / codec, **never** string-slicing HL7), keeping the rest of the
+  message parseable; the row is never deleted and a `documents_pruned` flag is set. All three
+  backends. (The ingest-time offload variant remains deferred.) (#47)
+- **Dual-control `config:reload` (ADR 0041 D2).** `config_reload` is now a gateable
+  `[approvals].operations` op — a **distinct** second approver must release a live config reload
+  (the requester can never self-approve; both identities land in the hash-chained audit). Opt-in /
+  deny-by-default, so single-operator deployments are unchanged. (#53)
+- **Startup code self-attestation (ADR 0041 D3).** At startup the engine hashes its loaded modules
+  against the wheel's `dist-info/RECORD`; on drift it records a hash-chained, off-box-teed
+  `startup_integrity` audit row and raises an alert (**alert-only by default**; opt-in
+  `[integrity].fail_closed_on_drift` refuses to start). A no-op on an editable (`pip install -e .`)
+  install, so development is never bricked. (#54)
+- **Operational-health metrics.** `GET /status` now meters the app-log directory's disk usage
+  alongside the database, and a per-connection **message-stall** alert rule fires when a connection's
+  oldest-undelivered age crosses a configurable threshold. (#50)
+
+### Changed
+- **Non-editable, hash-locked wheel is the enforced production default (ADR 0017 amendment).** The
+  prior recommendation is now the default for production deployments; editable installs remain a
+  no-op for development. (#54)
+
+### Fixed
+- **Intermittent Windows listener-teardown hang.** `MLLPSource` / `TcpSource` / `X12Source` no longer
+  `await server.wait_closed()` / `writer.wait_closed()` **unbounded** on the Windows Proactor loop
+  during teardown — a wait that never completes can no longer stall a shared event loop (the same
+  class as the resolved py3.11 hang). Added CI guards (a per-test `faulthandler` stack dump and a
+  step-level watchdog) so a future hang fails fast and names itself instead of silently timing out. (#55)
+
+### Docs
+- Refreshed `benchmarks/TUNING-BASELINE.md` with measured multi-process sharding throughput from the
+  Windows Server 2025 box (η ≈ 0.85 speedup shape; per-shard E_core ≈ 42 msg/s — a test-box SQLite
+  floor), plus the still-unmeasured hardware-gated follow-ups (enterprise E_core, the shared-DB
+  commit-wall sweep). (#28, #29)
+- Authored **ADR 0027** (per-connection retention) and **ADR 0042** (embedded-document pruning); added
+  EARS acceptance criteria to **ADR 0041** D2/D3; amended **ADR 0017** for the enforced wheel.
+
+### CI
+- Locked the smoke job's config directory (#603) and skipped a mirror-only Dependabot guardrail test
+  on the OSS mirror (#606), greening `main` CI post-0.2.8.
+
+## [0.2.8] — 2026-06-27 — Early Access
+
+A tooling/ops release: the load harness gains a **multi-shard driver** so one harness can drive a
+`supervise` cluster (unblocking the multi-core throughput measurement), `supervise` resolves
+`--env` files for its shards, and a prominent upgrade note for the config-directory permission
+guard introduced in 0.2.6.
+
+> ### ⚠ Upgrading from ≤ 0.2.5 — tighten config-dir ACLs first
+> The config-directory permission guard (SEC-003 / ADR 0036), added in **0.2.6**, refuses to load a
+> `--config` directory that is **writable by a broad principal** (e.g. `Authenticated Users` /
+> `S-1-5-11`). A deployment whose config dir inherits that write — common under `C:\srv\…` — will
+> **fail to start on first upgrade to ≥ 0.2.6** with *"refusing to load config from writable-by-others
+> path …"*. **Before upgrading**, tighten the directory (elevated):
+> ```powershell
+> icacls "<config-dir>" /inheritance:d /T
+> icacls "<config-dir>" /remove:g *S-1-5-11 /T          # drop Authenticated Users
+> icacls "<config-dir>" /grant *S-1-5-18:(OI)(CI)F /grant *S-1-5-32-544:(OI)(CI)F /T  # SYSTEM + Admins
+> ```
+> See [`docs/SERVICE.md`](docs/SERVICE.md) → *Update to a new build* and *Lock down the config
+> directory (CONFIG-2)*.
+
+### Added
+- **Multi-shard load driving (`messagefoundry-harness`).** `python -m harness` gains
+  **`--skip-preflight`** (drive shard MLLP ports that no single `--engine` owns) and a repeatable
+  **`--shard-engine <url>`**: the engine poller now takes a list of shard APIs and **sums** each
+  shard's `/stats` (read/written/backlog/in_pipeline/queue_depth/dead) into one cluster sample, so
+  the no-loss reconcile and drain are **cluster-aggregate** — a healthy K-shard run reports pass,
+  not a false "lost on intake". With no `--shard-engine` the behavior is byte-identical to before.
+  Two sample graphs ship for the throughput suite: `harness/config/store_once` (the
+  dedup-triggering one-handler-`list[Send]`-of-identical-body shape for store-once) and
+  `harness/config/passthrough` (an internal `PassThrough()` re-ingress hop); the load graph
+  (`harness/config/load`) is now shard-taggable via `MEFOR_LOAD_SHARD_ADT`/`_RESULTS`/`_OTHER`. (#604)
+
+### Fixed
+- **`supervise --project-root`.** `supervise` now accepts `--project-root` and forwards it to each
+  spawned `serve --shard`, so `supervise --config <dir> --env <env>` resolves each shard's
+  `environments/<env>.toml` (previously the shards resolved nothing from their spawned cwd and
+  required an explicit `--service-config` posture). Backward compatible — no `--project-root` is
+  unchanged. (#602)
+
 ## [0.2.7] — 2026-06-27 — Early Access
 
 A docs/packaging release that fixes the broken badge images on the PyPI project page
@@ -299,7 +396,8 @@ tests, but the external code review + penetration test (the bar for a security-c
 - Releases are built, SBOM'd (CycloneDX), and signed with [Sigstore](https://www.sigstore.dev/) — see the
   `release` workflow.
 
-[Unreleased]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.7...HEAD
+[Unreleased]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.8...HEAD
+[0.2.8]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.7...v0.2.8
 [0.2.7]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.6...v0.2.7
 [0.2.6]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.5...v0.2.6
 [0.2.5]: https://github.com/wshallwshall/MessageFoundry/compare/v0.2.4...v0.2.5

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: messagefoundry
-Version: 0.2.7
+Version: 0.2.9
 Summary: Open-source healthcare integration engine — route, transform, and validate messages across many formats and connection types
 Project-URL: Homepage, https://messagefoundry.org/
 Project-URL: Documentation, https://messagefoundry.org/

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/docs/CONFIGURATION.md

@@ -390,6 +390,7 @@ the rest are forward-compat placeholders for the future engine broker (accepted-
 |---|---|---|---|
 | `level` | enum | `info` | log level; never run prod at `debug` (PHI) — `serve` refuses it (Gate #1) |
 | `format` | enum | `text` | stdout rendering: `text` (default) or structured `json` (one object per line). Stdlib only — no structlog |
+| `log_dir` | str | _unset_ | the directory NSSM (or another supervisor) **rotates the engine's captured stdout/stderr into**. The engine never writes log **files** itself (it logs to stdout); set this only to tell it where the supervisor parks them, and `GET /status` then **meters that directory's total bytes + filesystem free space** alongside the DB metrics (#50). Unset = stdout-only, no metering. **Metadata only** — the file contents are never read. |
 | `forward_enabled` | bool | `false` | ship a copy of every record off-box to a syslog/SIEM collector (sec-offbox-log) so evidence survives a host compromise; requires `forward_host` |
 | `forward_host` | str | — | syslog/SIEM collector host (**required** when `forward_enabled`) |
 | `forward_port` | int | `514` | collector port (1–65535) |
@@ -412,12 +413,21 @@ so retention is opt-in.
 | `messages_days` | int | `0` | past N days, null inbound bodies (`raw`/`summary`/`error`) of **fully-resolved** messages (no `pending`/`inflight` delivery), keeping metadata. `0` = keep |
 | `dead_letter_days` | int | `0` | past N days, null the bodies of **dead-lettered** outbound rows (their own window — a dead row stays replayable until purged). `0` = keep |
 | `state_max_age_days` | int | `0` | past N days, **delete** transform-state entries (ADR 0005) last written before the cutoff — keeps the in-memory state cache + table bounded. A simple global age purge (by `set_at`); per-namespace policy is a follow-up. `0` = keep |
+| `connection_event_retention_hours` | int | `0` | past N **hours**, **delete** `connection_event` rows (the `[diagnostics]` #46 transport/lifecycle log — high-volume under a connect-per-message sender or a probe storm, so its own short window in **hours**, not days). `0` = inherit the `messages_days` body window (the ADR 0021 §7.5 default). |
 | `audit_days` | int | `0` | **reserved / not enforced.** The `audit_log` is a tamper-evident hash chain and HIPAA expects ~6-year retention, so audit is **keep-forever by design**; archive-first pruning is a tracked follow-up. Accepted so a forward-looking file still loads |
 | `max_db_mb` | int | `0` | advisory only: warn (WARNING log + an `AlertSink` `storage_threshold` event) when the DB (+ `-wal`/`-shm`) exceeds this. Never auto-deletes. `0` = off |
 | `purge_interval_seconds` | float | `3600` | how often the purge/maintenance loop runs a pass |
 | `wal_checkpoint_seconds` | float | `0` | `PRAGMA wal_checkpoint(TRUNCATE)` cadence (SQLite). `0` = off (rely on auto-checkpoint). Evaluated once per pass, so a value below `purge_interval_seconds` is effectively rounded up to it |
 | `vacuum_at` | str | `""` | daily local `"HH:MM"` to run `VACUUM` (SQLite; reclaims space freed by purges). `""` = off. A daily off-peak time, **not** a cron expression (no new dependency); VACUUM holds a write lock on the whole DB while it runs |
 
+> **Per-connection overrides ([ADR 0027](adr/0027-per-connection-retention.md)).** `messages_days` and
+> `dead_letter_days` are **global defaults** an individual connection may override: an **inbound** sets its
+> own `messages_days`, an **outbound** its own `dead_letter_days` (both `None` = inherit this global window,
+> `0` = keep that connection's bodies forever, `>0` = days). An inbound may also opt into **embedded-document
+> pruning** (`prune_documents_after` + `prune_documents_min_bytes`, [ADR 0042](adr/0042-embedded-document-pruning.md))
+> to strip bulky base64 attachments while keeping the readable message. These live on the connection (code-first
+> or in `connections.toml`) — see [CONNECTIONS.md](CONNECTIONS.md).
+
 > **SQLite-only.** Retention/maintenance runs on the SQLite backend. On the SQL Server
 > backend it is a DBA concern (TDE + a SQL Agent purge/shrink job). Each pass that does real work
 > writes one `retention_purge` `audit_log` entry (cutoffs + counts, **no** message content). Design:
@@ -432,6 +442,7 @@ so retention is opt-in.
 | `internal_error` | enum | `continue` | what a delivery worker does on an **internal/code error** (a non-`DeliveryError` exception from `send` — our bug, not the partner's): `continue` (dead-letter the row + advance) or `stop` (halt the connection's worker, preserve the message for replay, raise a `connection_stopped` alert). Per-outbound `internal_error=` overrides. Partner NAKs / transport failures are unaffected. |
 | `buildup_max_depth` | int | _unset_ | raise a `queue_buildup` alert when an outbound lane's pending depth reaches this. Unset = depth dimension off (a healthy ceiling is throughput-specific, so there's no safe default). Per-outbound `buildup=BuildupThreshold(...)` overrides. |
 | `buildup_max_oldest_seconds` | num | 300 | raise `queue_buildup` when the lane's **oldest** pending message has waited this long (a stuck/retry-forever head is the classic cause). On by default — a head stuck >5 min is a problem in any environment. Set to unset/`0`-disable via a per-outbound override. |
+| `stall_max_oldest_seconds` | num | _unset_ | raise a `message_stall` alert (Corepoint "Max Message Stall", [ADR 0014](adr/0014-alert-routing-rules.md)) when an outbound lane's **oldest undelivered message** has waited this long. **Unset (the default) = the stall alert is OFF** — deny-by-default/opt-in, because it overlaps `buildup_max_oldest_seconds`'s age dimension and would double-page if both fired. Set a threshold to turn it on; a per-outbound `stall=StallThreshold(...)` overrides it. The stall event routes through `[[alerts.rules]]` like any other ([ADR 0014](adr/0014-alerting-rules-engine.md)). |
 | `outbox_workers` | int | per-outbound | delivery concurrency (planned) |
 | `dead_letter` | enum | `keep` | `keep`/`drop`-after-N (planned) |
 
@@ -440,6 +451,22 @@ so retention is opt-in.
 |---|---|---|---|
 | `max_correlation_depth` | int (≥1) | 8 | **Re-ingress loop cap** (ADR 0013 Increment 2). When a captured reply is re-ingressed (`reingress_to=`/`Loopback()`), the re-ingressed message carries a `correlation_depth`; a message at this depth still routes, but the next hop (depth+1) **dead-letters** its re-ingress work-row and marks the origin `ERROR`. Coarse by design — it bounds *total work*, not topology, so a chain that legitimately bounces A→B→A a few times needs headroom. 8 is safe for typical request→response→route feeds; raise it for deep correlation chains, lower it to fence a misbehaving loop. (A value of 0 would dead-letter every re-ingress, so the floor is 1.) |
 
+### `[diagnostics]`
+The Corepoint-style **event log** (#46) — a metadata-only record of connection lifecycle / pre-ingress
+failures and the ACK/NAK the engine returns. **Both master switches are on by default and safe to be:**
+they store only non-PHI metadata (connection name, peer IP, a scrubbed reason, the ACK disposition), and
+the AA-ACK *body* is stored only when the store is encrypted (else NULL); a NAK body is never persisted.
+A per-connection `capture_connection_errors` / `capture_ack` flag overrides the matching master switch for
+one connection (see [CONNECTIONS.md](CONNECTIONS.md)).
+
+| Key | Type | Default | Notes |
+|---|---|---|---|
+| `connection_events` | bool | `true` | master switch for the **connection/transport event log**: inbound lifecycle (established/closed) + pre-ingress failures (allowlist/capacity/oversize/peer-reset/framing) + outbound lane transitions (connection_lost/restored). Metadata-only, written off the hot path by a drain task. Per-connection `capture_connection_errors` overrides it. |
+| `response_sent` | bool | `true` | master switch for **"Response Sent"** — the ACK/NAK returned to an inbound sender. Always captures the disposition metadata (`ack_code`/`phase`/`outcome`); the AA body is stored only on an encrypted store, and every NAK body is NULL. Per-connection `capture_ack` overrides it. |
+
+> Retention for the event log has its own short window — `[retention].connection_event_retention_hours`
+> (in **hours**; `0` = inherit `messages_days`).
+
 ### `[egress]`
 Fail-closed **outbound destination allowlist** (WP-11c; ASVS 13.2.4/13.2.5/14.2.3) — bounds where the
 engine may **send** PHI, so a fat-fingered or hostile destination can't exfiltrate it. Each list is
@@ -484,7 +511,8 @@ bytes/SQL leave the box) and retains the would-send payload for parity compariso
 > the message just finalizes `PROCESSED`.
 
 ### `[alerts]`
-Where the delivery pipeline's operational alerts (`connection_stopped`, `queue_buildup`) are
+Where the delivery pipeline's operational alerts (e.g. `connection_stopped`, `queue_buildup`,
+`connection_error`, `message_stall`, `integrity_drift`) are
 delivered. **Both transports are off by default** — with neither configured, events are logged at
 `WARNING` (the `LoggingAlertSink`). A transport turns on when its essentials are present. Payloads
 carry the connection name + queue shape only — **never a message body** (no PHI). Delivery is
@@ -515,7 +543,7 @@ silences an event you didn't name. Matching is pure config (no code/`eval`).
 
 | Key | Type | Default | Notes |
 |---|---|---|---|
-| `event_type` | str | `any` | match this event: `any`, `connection_stopped`, `queue_buildup`, `storage_threshold`, or `cert_expiry` |
+| `event_type` | str | `any` | match this event: `any`, `connection_stopped`, `queue_buildup`, `storage_threshold`, `cert_expiry`, `connection_error`, `message_stall`, or `integrity_drift` |
 | `connection` | str (glob) | `*` | glob over the connection name (e.g. `OB_*`, `IB_ACME_*`) |
 | `min_depth` | int | _unset_ | `queue_buildup` only — match only when pending depth is at/over this |
 | `min_oldest_seconds` | num | _unset_ | `queue_buildup` only — …or the oldest pending message has waited at least this long |
@@ -708,6 +736,21 @@ Optional **dual-control (maker-checker)** approval for high-value actions (ASVS
 | `operations` | list[str] | `["connection_purge", "dead_letter_replay"]` | which operations require approval; each must be a known op key (a typo is refused at startup) |
 | `expiry_hours` | num | 72 | a pending request can no longer be approved after this many hours (`0` = never expires) |
 
+### `[integrity]`
+Startup **self-attestation of the installed engine wheel** ([ADR 0041](adr/0041-load-path-attestation-and-change-attribution.md)
+D3) — a runtime in-place-tamper tripwire (`messagefoundry/integrity.py`). At startup (and on demand) the
+engine hashes every **loaded** first-party `messagefoundry` module file against the installed wheel's
+`*.dist-info/RECORD` baseline; on **drift** (a loaded module no longer matching its RECORD hash) it records
+a hash-chained `startup_integrity` audit row and fires the `AlertSink`. It complements ADR 0036 (which guards
+the *config dir*) by covering the installed *site-packages* an admin with venv-write + restart rights could
+edit in place. Both keys default **safe**: attestation is on but **alert-only** (it never blocks startup), and
+an **editable** install (`pip install -e .` — no RECORD baseline) is a **no-op**, so dev is never bricked.
+
+| Key | Type | Default | Notes |
+|---|---|---|---|
+| `enabled` | bool | `true` | run startup attestation at all. On by default (alert-only is harmless); a **no-op** off an editable install. Set `false` only to suppress the check entirely (e.g. an unusual packaging where RECORD is known-stale) — you then lose the in-place-tamper tripwire. |
+| `fail_closed_on_drift` | bool | `false` | when `true`, drift makes `serve` **refuse to start** (after recording the audit row + alerting). Default `false` = **alert-only**: a legitimate reviewed in-place security hotfix (the documented vendored-parser patch contingency) would itself trip a RECORD mismatch, so fail-closed-by-default would brick a legitimate patch. Opt in for hard enforcement on a locked-down instance. |
+
 ### `[engine]`
 | Key | Type | Default | Notes |
 |---|---|---|---|

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/docs/CONNECTIONS.md

@@ -173,7 +173,8 @@ collision.
 | `tls_check_hostname` | out | `true` | require the server cert to match `host` (SNI + hostname check). |
 
 Plus on `inbound(...)`: `ack_mode` (`original`/`enhanced`/`none`), `strict`, `hl7_version`. On
-`outbound(...)`: `retry` (`RetryPolicy`), `ordering`, `internal_error`, `buildup`, and `simulate`
+`outbound(...)`: `retry` (`RetryPolicy`), `ordering`, `internal_error`, `buildup`, `stall`
+(`StallThreshold` — Corepoint "Max Message Stall", #50; off unless set, see below), and `simulate`
 (`bool`, default `false`). `simulate=True` puts the outbound in **shadow / parallel-run mode** (#15): it
 runs the full transform + count-and-log and finalizes the message `PROCESSED`, but **suppresses the real
 egress** (no bytes/SQL leave the box) and retains the would-send payload for parity comparison — so a
@@ -1011,6 +1012,90 @@ outbound("MLLP-OUT_PAYER_ELIG", MLLP(host=env("payer_host"), port=2575, reingres
 # a Handler Sends the eligibility query to MLLP-OUT_PAYER_ELIG; its reply re-ingresses into IB-LOOP_PAYER_ELIG.
 ```
 
+## Per-connection retention, document pruning & diagnostics overrides
+
+A connection may **override** several service-wide `[…]` defaults for just itself. Each is set the same
+two ways as `retry`/`buildup` — **code-first** on `inbound(...)`/`outbound(...)`, **or** as a key in
+`connections.toml` (ADR 0007) — and each defaults to **inherit the global setting** when omitted.
+
+### Retention overrides ([ADR 0027](adr/0027-per-connection-retention.md))
+
+Override the global `[retention]` body-null windows per connection. `None` (omitted) = inherit the global
+window; `0` = keep this connection's bodies **forever**; `>0` = days.
+
+| Key | Dir | Type | Default | Meaning |
+|-----|-----|------|---------|---------|
+| `messages_days` | in | int | inherit `[retention].messages_days` | past N days, null this **inbound's** received message bodies (keyed on the receiving inbound), keeping the metadata row. `0` = keep forever |
+| `dead_letter_days` | out | int | inherit `[retention].dead_letter_days` | past N days, null the bodies of **this outbound's** dead-lettered rows (keyed on the outbound that dead-lettered them). A dead row stays replayable until its body is purged. `0` = keep forever |
+
+### Embedded-document pruning ([ADR 0042](adr/0042-embedded-document-pruning.md), #47)
+
+A separate **inbound** lever that evicts only the bulky base64 **embedded document** (a `mfb64:v1:`
+carriage value / an HL7 `OBX-5` ED embed) **in place** to a small tombstone — keeping the surrounding,
+readable message — distinct from `messages_days`, which nulls the **whole** body.
+
+| Key | Dir | Type | Default | Meaning |
+|-----|-----|------|---------|---------|
+| `prune_documents_after` | in | int | `None` = **never prune** (back-compat) | after N **days**, strip each embedded document for this inbound. Must be `> 0` |
+| `prune_documents_min_bytes` | in | int | `None` = strip **any** size | skip an embed whose decoded size is **below** this byte threshold (keep small embeds, evict only the bulky ones). Setting it **requires** `prune_documents_after` (else a wiring error) |
+
+### Diagnostics / event-log overrides ([ADR 0021](adr/0021-inbound-ack-nak-capture-response-sent.md), #46)
+
+Override the `[diagnostics]` master switches for one connection. **Tri-state:** omitted = inherit the
+matching master switch; `true`/`false` = explicit per-connection override.
+
+| Key | Dir | Type | Default | Meaning |
+|-----|-----|------|---------|---------|
+| `capture_ack` | in | bool | inherit `[diagnostics].response_sent` | record the **"Response Sent"** ACK/NAK metadata for this inbound (the AA body only on an encrypted store; a NAK body is never stored) |
+| `capture_connection_errors` | in | bool | inherit `[diagnostics].connection_events` | record this connection's **lifecycle + pre-ingress failure** events (established/closed, allowlist/capacity/oversize/peer-reset/framing) |
+
+### `stall` — Max Message Stall ([ADR 0014](adr/0014-alerting-rules-engine.md), #50)
+
+An **outbound** override of the `[delivery].stall_max_oldest_seconds` global: raise a `message_stall`
+alert when this lane's **oldest undelivered message** has waited too long.
+
+| Key | Dir | Type | Default | Meaning |
+|-----|-----|------|---------|---------|
+| `stall` | out | `StallThreshold` | inherit `[delivery]` (off unless set) | `StallThreshold(max_oldest_seconds=…)` — `None` keeps the stall alert **off** (it overlaps `buildup`'s age dimension, so it's opt-in to avoid double-paging). In `connections.toml` it is an `[outbound.stall]` table with `max_oldest_seconds` (see the example below) |
+
+```python
+from messagefoundry import MLLP, inbound, outbound
+from messagefoundry.config.models import StallThreshold
+
+# Inbound: keep this feed's bodies only 7 days, and prune embedded documents >256 KiB after 1 day.
+inbound("IB_ACME_RAD", MLLP(port=2576), router="rad_router",
+        messages_days=7, prune_documents_after=1, prune_documents_min_bytes=256 * 1024,
+        capture_ack=True)                         # force-capture the ACK even if the master switch is off
+
+# Outbound: keep this destination's dead-letter bodies 90 days; alert if a message stalls >10 min.
+outbound("OB_PACS_RAD", MLLP(host="pacs", port=11112),
+         dead_letter_days=90, stall=StallThreshold(max_oldest_seconds=600))
+```
+
+```toml
+# connections.toml — the same overrides as data.
+[[inbound]]
+name = "IB_ACME_RAD"
+transport = "mllp"
+router = "rad_router"
+messages_days = 7
+prune_documents_after = 1
+prune_documents_min_bytes = 262144
+capture_ack = true
+  [inbound.settings]
+  port = 2576
+
+[[outbound]]
+name = "OB_PACS_RAD"
+transport = "mllp"
+dead_letter_days = 90
+  [outbound.settings]
+  host = { env = "pacs_host" }
+  port = 11112
+  [outbound.stall]
+  max_oldest_seconds = 600
+```
+
 ## Resource management & limits (ASVS 13.1.2 / 13.1.3 / 13.2.6)
 
 How the engine bounds connections, threads, and retries per external system, and what happens **when a

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/docs/SERVICE.md

@@ -91,6 +91,16 @@ definition drifted — the install script is idempotent (it stops and reconfigur
 If the package was installed **non-editable** (a plain `pip install .`), the venv holds a
 snapshot of the old code — run `.venv\Scripts\python.exe -m pip install -e .` first, then restart.
 
+> **⚠ Upgrading from ≤ 0.2.5 → ≥ 0.2.6: tighten the config-dir ACLs first.** The config-directory
+> permission guard (SEC-003 / ADR 0036) did **not** exist in 0.2.5. From 0.2.6 on, `serve` refuses to
+> start against a `--config` directory **writable by a broad principal** (e.g. `Authenticated Users` /
+> `S-1-5-11`) — *"refusing to load config from writable-by-others path …"*. A config dir that inherits
+> that write (common under `C:\srv\…`) therefore **fails its first start after the upgrade**. Lock it
+> down **before** restarting onto the new build — use the surgical recipe under
+> [Lock down the config directory (CONFIG-2)](#lock-down-the-config-directory-config-2)
+> (`icacls /inheritance:d /T` + `/remove:g *S-1-5-11 /T` + grant SYSTEM/Admins), the lighter-touch
+> alternative to a full `/inheritance:r` reset for a shared tree.
+
 ## Start / stop / status
 
 ```powershell
@@ -190,6 +200,15 @@ trust boundary: anyone who can write a `.py` file there can run code as the serv
   (full) and the service account (read+execute). It is **opt-in** because the config dir often lives
   inside a developer's repo where stripping inheritance is surprising — for production, point
   `-Config` at a dedicated admin-owned directory and pass `-LockConfigDir`.
+- **Fix an existing tree that inherits a broad write grant** without a full ACL reset. A config dir
+  placed under a shared root (e.g. `C:\srv\…`) often inherits `Authenticated Users` (`S-1-5-11`)
+  write, which trips the guard below. The lighter-touch alternative to `/inheritance:r` is to break
+  inheritance, surgically drop just the broad principal, and grant the run-as user read+execute:
+  ```powershell
+  icacls "C:\srv\mefor\config" /inheritance:d /T
+  icacls "C:\srv\mefor\config" /remove:g *S-1-5-11 /T
+  icacls "C:\srv\mefor\config" /grant "<run-as-user>:(OI)(CI)RX" /T
+  ```
 - The loader **actively enforces** this at load time (and on `/config/reload`), not just as a
   documented recommendation (ADR 0036, SEC-003):
   - On **Windows** the loader now parses the directory's and each `*.py`'s NTFS owner + DACL and

{messagefoundry-0.2.7 → messagefoundry-0.2.9}/docs/adr/0017-consumer-deployment-model.md

@@ -215,3 +215,42 @@ installs use the pinned wheel. That migration already had to repurpose the built
   the security-posture tier — both Blocker rows must land together.
 - Most of the model already works; the remaining effort is **packaging, documentation, scaffolding, one
   env-name lift, and a few isolation/drift guards** — not a redesign of the config or value layers.
+
+## Amendment (2026-06-27) — non-editable wheel from recommendation to enforced production default
+
+> Append-only amendment from the **insider-code-tampering review** (see
+> [ADR 0041](0041-load-path-attestation-and-change-attribution.md) D3, BACKLOG #54). It **does not rewrite**
+> the accepted three-tier model or the eight ratified decisions above — it **tightens** Ratified Decision 5
+> ("Boundary enforcement → packaging-primary") from a documented *recommendation* into an **enforced
+> production default**.
+
+**What changes.** The original decision said the non-editable wheel is the *recommended* boundary and an
+optional Windows venv ACL-lock is *defense-in-depth*. ADR 0041 D3 adds a **runtime tripwire** that converts
+this from advice into an enforced posture:
+
+- The **non-editable, hash-locked wheel is the production default.** Editable `pip install -e .` is
+  **retired from production deployment docs** ([EARLY-ADOPTER-GUIDE.md](../EARLY-ADOPTER-GUIDE.md),
+  [DEPLOYMENT.md](../DEPLOYMENT.md), SERVICE.md): editable installs are documented **only** for engine
+  co-development, never for a deployed instance. The two-repo consumer model (pinned wheel + org config repo)
+  is now the single documented production install path.
+- Enforcement is **startup self-attestation** (ADR 0041 D3): the engine hashes its loaded `messagefoundry`
+  module files against the wheel's `*.dist-info/RECORD` baseline at boot. On drift the **default is
+  alert-only** (a `startup_integrity` audit row + `AlertSink`); an **opt-in** `[integrity].fail_closed_on_drift`
+  hard-refuses to start. This makes the read-only-engine boundary **detectable at runtime**, not merely
+  enforced by file packaging.
+
+**Implemented as a no-op on editable dev installs.** An editable install has **no `dist-info/RECORD`**
+baseline to attest against, so the tripwire is a **no-op/advisory** there — `pip install -e ../MessageFoundry`
+during co-development (the workflow named in *First instance of the pattern* above) is **byte-for-byte
+unchanged** and never bricked. The enforcement applies precisely where it should: a **deployed, wheel-installed
+production instance**.
+
+**Why the default is alert-only, not fail-closed.** The documented vendored-patch contingency for the dormant
+`python-hl7`/`hl7apy` parsers (a legitimate, reviewed security hotfix applied in place) would itself trip a
+`RECORD` mismatch. A fail-closed-by-default posture would therefore **brick a legitimate security patch** at
+the worst moment. Alert-only-by-default detects-and-attributes the drift while keeping the service up;
+operators who want hard enforcement opt in via `[integrity].fail_closed_on_drift`.
+
+**What this does not change.** The accepted three-tier ownership model, the same-commit-everywhere promotion
+model (Decision 4), the env-name decoupling Blockers, and the AGPL/licensing posture (Decision 6) are all
+unchanged. This amendment narrows **only** the install/boundary-enforcement surface of Decision 5.