instar 1.3.526 → 1.3.528

package/upgrades/1.3.528.md

@@ -0,0 +1,34 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+The **relationship registry is now the SECOND concrete consumer of the HLC replicated-store foundation and the FIRST PII kind** — `relationship-record` rides the foundation primitives (envelope / union-reader / conflict-store / rollback-unmerge / bounds) so a person the agent knows on one machine is known on the others. Per `docs/specs/ws23-relationships-userregistry-security.md` (§5 merge semantics, §4.2 tombstones, §4.3 right-to-erasure, §7 tests).
+
+- **The `relationship-record` replicated kind** (`src/core/RelationshipsReplicatedStore.ts`) — a STRICT typed schema that is a **discriminated union on `op`** (a value schema AND a delete-tombstone schema coexist under one kind, REQ-D6) and **type-clamps every known field on receive** (`firstInteraction`/`lastInteraction` are ISO-8601-only, `interactionCount`/`significance` are finite numbers, free text is length-clamped — so a foreign, attacker-controlled record can't smuggle markup through a render slot that bypasses `sanitize()`, REQ-M3 gap #4/#8). The **disclosure-minimized projection** (REQ-M4) emits ONLY the enumerated resolution + merge-relevant fields — never the raw on-disk blob and never the local UUID `id`. The cross-machine `recordKey` is the **channel-set identity surface** (REQ-D17), derived deterministically from a person's sorted channel-uids, so the same human reaches the same record across machines even though each mints its own UUID. The per-entry cap is **raised to 64KB** for this PII kind (REQ-M3 gap #10) so a fat relationship replicates instead of wedging the stream; a record still over-cap after projection is a NAMED rejection, never a silent truncate. HIGH impact tier (append-both-and-flag, never a silent clobber of two divergent people).
+- **DUAL REGISTRY** — `relationship-record` is registered in BOTH `JOURNAL_KINDS` (`CoherenceJournal.ts` — the static serve/apply/advert half, with a `DEFAULT_RETENTION` entry that is never `rotateKeep:0` for compliance) AND `ReplicatedKindRegistry` (the dynamic half). A kind in only one silently replicates nothing; the CI ratchet asserts the coupling.
+- **Emit-on-mutation funnel + tombstones** — `RelationshipManager` routes every persistence mutation through its single `save()` funnel and every deletion through `delete()`/`mergeRelationships()`; an injected (dark-by-default) replication emitter emits a `put` on save and a channel-keyed `op:'delete'` **tombstone** on delete (so an erased person stays erased even on a machine that was offline at delete time, REQ-D4/§4.3). `mergeRelationships` emits a coherent put(survivor)+delete(merged) pair whose recordKeys are DISTINCT — no dangling tombstone, no replication loop.
+- **Read-only neutralized union** — the peer-read surface ("what do my OTHER machines know about this person") resolves THROUGH the bypass-proof `ReplicatedStoreReader` and renders each foreign record inside a `<replicated-untrusted-data origin="…">` envelope (quoted data, never an instruction). It is DISTINCT from the local-authoritative `resolveByChannel`/`getContextForPerson` — identity RESOLUTION of an inbound principal stays local-only (REQ-M14).
+- **Config + advert + awareness + migration** — `multiMachine.stateSync.relationships { enabled:false, dryRun:true }` added to ConfigDefaults (classified in `DARK_GATE_EXCLUSIONS`; the dark-gate line-map recomputed by hand; `applyDefaults` backfills existing agents); the `stateSyncReceive` advert self-reports `relationships` from the registry; the "One Memory" CLAUDE.md section gains a WS2.3 PII line (with the at-rest honesty note) in both `generateClaudeMd` and an idempotent `migrateClaudeMd` splicer (framework shadow markers already cover the section).
+- **Slice** — this PR builds `relationship-record` ONLY. `user-registry` + `topic-operator` (the spec's other two PII kinds) reduce to "add schema + projection + flag" on this proven machinery and are a tracked follow-up (CMT-1416).
+
+Pure MECHANISM, dark by default. A single-machine / flag-off agent is a strict no-op (no PII ever crosses a machine boundary while dark).
+
+## What to Tell Your User
+
+None while dark — internal substrate. The user-visible capability — a person I know on one machine is known on your others — becomes real only when an operator turns on cross-machine relationship replication, and it carries an **honest at-rest trade**: while on, every machine in the pool (including any cloud VM the operator rents but doesn't physically control) keeps a plaintext copy of everyone the agent knows under that machine's filesystem permissions, NOT the encrypted vault (the connection between machines IS encrypted, so nobody reads it in transit). That honesty ships in the CLAUDE.md "One Memory" section so the agent surfaces it the moment a user asks "is my contact data shared / encrypted on the other machine?".
+
+## Summary of New Capabilities
+
+None user-facing while dark. New internal module `RelationshipsReplicatedStore.ts`; `RelationshipManager` gains injected (dark) replication-emit + union-read seams. No new routes (the foundation `/state/conflicts` · `/state/resolve-conflict` · `/state/quarantine` surface is reused).
+
+## Evidence
+
+- `tests/unit/RelationshipsReplicatedStore.test.ts` (35) — dual-registry coupling; recordKey identity derivation (no split identity, no stranger-collision, not the UUID); disclosure-minimization (no local id, no extra field); `fat-record-replicates` + `fat-record-does-not-wedge-stream`; `foreign-record-type-clamped` (ISO-8601 / finite-number clamps reject smuggled markup; unknown field dropped; freetext clamped); `tombstone-coexists-with-value` (the op:'delete' schema branch); the HIGH-impact append-both union merge; foreign-record render safety. Green.
+- `tests/unit/relationship-replication-emit.test.ts` (6) — emit-on-every-mutation; DARK no-op; delete emits a channel-keyed tombstone; `mergeRelationships` put+delete coherence (distinct recordKeys); a throwing emitter never breaks the local write. Green.
+- `tests/unit/relationship-union-read.test.ts` (5) — through the REAL `ReplicatedStoreReader`: union-reader-cannot-be-bypassed (disabled = no-op); append-both; `erasure-reaches-offline-peer` (tombstone wins over a stale value); post-unmerge zero-dangling. Green.
+- `tests/unit/ws23-relationships-wiring.test.ts` (9) — dual-registry + server.ts registration/union-reader/peer-read-seam wiring + ConfigDefaults dark default + the awareness section. Green.
+- `tests/integration/ws23-relationships-routes.test.ts` (3) — the relationship routes still work with the replication funnel attached; a route-driven DELETE fires the channel-keyed tombstone through the funnel. Green.
+- `tests/e2e/ws23-relationships-alive.test.ts` (3, Phase 1) — the WS2.3 path is ALIVE on the real AgentServer boot path with the REAL relationship-record schema registered: an enabled relationship-record conflict is open + readable + resolvable over HTTP (200), disabled returns 503, Bearer auth required. Green.

package/upgrades/side-effects/ws21-preferences-replicated-store.md

@@ -0,0 +1,91 @@
+# Side-Effects Review — WS2.1 preferences replicated store
+
+**Version / slug:** `ws21-preferences-replicated-store`
+**Author:** `echo`
+**Second-pass reviewer:** `not required`
+**Spec:** `docs/specs/multi-machine-replicated-store-foundation.md` (converged + approved)
+
+## Summary of the change
+
+Wire the auto-learned preference store (`PreferencesManager`, `.instar/preferences.json`)
+onto the HLC replicated-store foundation as the FIRST concrete `pref-record` replicated
+kind. A preference learned on one machine is honored on the others — read through the
+no-clobber union with operator-resolved conflicts. New module
+`src/core/PreferencesReplicatedStore.ts` holds the store schema, impact tier (high),
+per-kind bounds, the emit-envelope builder, and the load-bearing union-aware read
+(`mergeUnionToPreferences` / `buildUnionSessionContext`). `pref-record` is registered in
+BOTH `JOURNAL_KINDS` (CoherenceJournal — static half) AND `ReplicatedKindRegistry`
+(dynamic half). The `/preferences/session-context` route consults a wired
+`ReplicatedStoreReader` ONLY when `multiMachine.stateSync.preferences.enabled`. Ships
+DARK (`enabled:false`, `dryRun:true`).
+
+## Decision-point inventory
+
+1. **Impact-tier resolution (spec §15.1):** preferences are HIGH-impact at the
+   replication layer (append-both-and-flag), but the consumer read injects BOTH
+   variants as advisory hints on an open conflict — never blocking. Decision recorded
+   in the build prompt + the spec; the conservative side (never silently clobber).
+2. **Coordination with the legacy PreferencesSync (CMT-1416):** the foundation path
+   SUPERSEDES the seamlessness path; both ship dark → zero runtime duplication. The
+   route gives the foundation path precedence when its flag is on.
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject?** The pref-record store schema
+rejects a record with an empty `learning` or a path-shaped `provenance` (the §4 jail).
+Both are malformed-data rejections at the RECEIVE door (a replicated record from a
+peer), never a user-initiated action — a local `recordPreference` is unaffected. With
+the flag OFF (the default), nothing is emitted, validated, or read through the union:
+strict no-op, byte-identical single-machine behavior. The route's foundation branch is
+gated on `stateSync.preferences.enabled` so a dark agent keeps its exact legacy read.
+
+## 2. Under-block
+
+**What does this still miss?** The `loadOriginRecords` seam reads only the OWN
+preference store today — peer `pref-record` replicas land via the journal apply path in
+a later rollout stage. So on a single-machine / pre-apply agent the union is a no-op
+(just the local record). This is the honest dark-ship boundary, not a gap: the union /
+conflict / both-variants logic is fully exercised by unit tests against synthetic
+multi-origin inputs. The `dryRun` mode (log intended merges without mutating) is the
+config ladder's middle rung; the emit/apply transport wiring is gated by the same flag.
+
+## 3. Level-of-abstraction fit
+
+**Right layer?** Yes. The new kind rides the existing foundation primitives
+(ReplicatedRecordEnvelope / UnionReader / ConflictStore / RollbackUnmerge /
+ReplicationBudget / StoreSnapshot) — no foundation change beyond registering the kind in
+the dual registry. The bypass-proof read goes through `ReplicatedStoreReader` (the
+single funnel), so no caller can read the raw store around the no-clobber rule. The
+consumer-specific concerns (the dedupeKey↔recordKey mapping, the local-only
+`violationPattern` strip, the both-variants reconciliation) live in one module.
+
+## 4. Reversibility
+
+Fully reversible. The whole feature is dark behind `multiMachine.stateSync.preferences`
+(default `enabled:false`). Disabling it is a strict no-op (the route keeps its legacy
+path; no emission, no union read). A peer's contribution can be rolled back via the
+existing RollbackUnmerge (`/state/quarantine`), which the foundation already wires. No
+destructive operation is introduced; the conflict ledger + dropped-origins set are the
+foundation's existing durable state under `.instar/state/state-sync/`.
+
+## 5. Migration / fleet-rollout surface
+
+- **Config:** `multiMachine.stateSync.preferences { enabled:false, dryRun:true }` added
+  to `ConfigDefaults` — `applyDefaults` add-missing backfills existing agents on update
+  (Migration Parity; no bespoke migrateConfig block needed). Classified in
+  `DARK_GATE_EXCLUSIONS` (optional-integration, staged rollout) so the literal
+  `enabled:false` is accounted for by the dark-gate lint.
+- **CLAUDE.md awareness:** the existing "One Memory (replicated stores)" section gains a
+  WS2.1 preferences-consumer line in BOTH `generateClaudeMd` (new agents) and
+  `migrateClaudeMd` (existing agents, via an idempotent content-sniff splicer guarded by
+  the unique "Preferences are the FIRST live store" marker).
+- **Dual-registry CI ratchet:** the `ReplicatedRecordEnvelope.test.ts` coupling test now
+  asserts `pref-record` is coupled in both registries — a future edit removing it from
+  `JOURNAL_KINDS` would RED.
+
+## Verdict
+
+Safe to ship dark. Zero behavior change for any agent until an operator flips
+`multiMachine.stateSync.preferences.enabled`. The advisory both-variants read is the
+load-bearing correctness property and is proven by a §12 wiring test (an open conflict
+never suppresses a usable hint).

package/upgrades/side-effects/ws23-relationships.md

@@ -0,0 +1,150 @@
+# Side-Effects Review — WS2.3 relationships (first PII replicated store)
+
+**Version / slug:** `ws23-relationships`
+**Author:** `echo`
+**Second-pass reviewer:** `not required (dark-ship, observe-only until flag flip)`
+**Spec:** `docs/specs/ws23-relationships-userregistry-security.md` (converged + approved, CMT-1413)
+
+## Summary of the change
+
+Wire the relationship registry (`RelationshipManager`, `relationships/<id>.json`) onto
+the HLC replicated-store foundation as the SECOND concrete consumer and the FIRST PII
+kind, `relationship-record`. New module `src/core/RelationshipsReplicatedStore.ts` holds
+the strict typed schema (discriminated union on `op` for value + tombstone, with
+per-field type-clamps), the disclosure-minimized projection, the channel-set identity
+recordKey derivation, the HIGH impact tier, per-kind bounds, the 64KB per-entry cap,
+the tombstone builder, the union-aware read, and the foreign-record render-safety
+helper. `relationship-record` is registered in BOTH `JOURNAL_KINDS` (CoherenceJournal —
+static half) AND `ReplicatedKindRegistry` (dynamic half). `RelationshipManager` gains
+two injected, dark-by-default seams: a replication emitter (a `put` on the save funnel,
+a channel-keyed `op:'delete'` tombstone on delete/merge) and a union-read seam (the
+read-only neutralized peer-context surface). Ships DARK
+(`multiMachine.stateSync.relationships { enabled:false, dryRun:true }`).
+
+## Decision-point inventory
+
+1. **recordKey = channel-set identity surface, NOT the local UUID (REQ-D17).** The on-disk
+   `id` is per-machine; the same human has different UUIDs on different machines, so the
+   replicated key is a deterministic `sha256(sorted(channel-uids))`. This is the
+   single most load-bearing identity decision; it is documented in the module and proven
+   by the recordKey tests (same person → same key regardless of order; disjoint people →
+   different keys; channel-less → null, not replicable).
+2. **64KB per-entry cap (REQ-M3 gap #10).** A fat relationship exceeds the default 8KB
+   applier cap; raised to 64KB so the highest-PII records replicate, with a NAMED
+   over-cap rejection (`RelationshipRecordTooLargeError`) rather than silent truncate or
+   suspect-wedge.
+3. **mergeRelationships put+delete coherence (the sharp edge).** A local merge emits a
+   survivor `put` (now carrying the consolidated channel set) AND a tombstone keyed on
+   the merged record's OLD standalone channel set. The two recordKeys are DISTINCT (the
+   survivor subsumes the merged's channels), so the tombstone can never suppress the
+   survivor — no dangling tombstone, no replication loop. Proven by the merge test.
+4. **Slice (CMT-1416).** Only `relationship-record` ships here; user-registry +
+   topic-operator are a tracked follow-up on the proven machinery.
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject?** The receive-side schema rejects a
+foreign record with a non-ISO-8601 `firstInteraction`/`lastInteraction`, a non-number
+`interactionCount`/`significance`, a missing/empty `name`, a bad channel, or > MAX_CHANNELS
+channels — all malformed-data rejections at the RECEIVE door (a peer's replicated
+record), never a user-initiated local action. A local `findOrCreate`/`updateNotes`/… is
+unaffected (it goes through the normal save path; the emit is best-effort AFTER the
+durable local write and swallows its own failures). With the flag OFF (default), nothing
+is emitted, validated, or read through the union: strict no-op — NO PII ever crosses a
+machine boundary.
+
+## 2. Under-block
+
+**What does this still miss?** The `loadOriginRecords` seam reads only the OWN
+relationship store today — peer `relationship-record` replicas land via the journal
+apply path in a later rollout stage (the same staging as WS2.1's shipped reference). So
+on a single-machine / pre-apply agent the union is a no-op (just the local record); the
+union / conflict / append-both / tombstone-wins logic is fully exercised by unit tests
+against synthetic multi-origin inputs and through the real `ReplicatedStoreReader`. The
+spec's deeper PII obligations beyond this slice (operator-mandate-gated destructive
+local delete on receive, the deferred-erasure queue, the merge-skew gate, AAD binding)
+are tracked in the converged spec and ride later increments; this PR establishes the
+schema/projection/recordKey/tombstone/cap machinery they build on. The at-rest
+plaintext-rest exposure delta is the operator-accepted residual (REQ-A2) stated honestly
+in the CLAUDE.md awareness section.
+
+## 3. Level-of-abstraction fit
+
+**Right layer?** Yes. The new kind rides the existing foundation primitives
+(ReplicatedRecordEnvelope / UnionReader / ConflictStore / RollbackUnmerge /
+ReplicationBudget) — no foundation change beyond registering the kind in the dual
+registry. The bypass-proof read goes through `ReplicatedStoreReader` (the single
+funnel). The PII-specific concerns (the channel-set recordKey, the type-clamp schema,
+the disclosure-minimized projection, the tombstone, the untrusted-data render envelope)
+live in one module. The emit/read seams on `RelationshipManager` are injected so the
+manager stays pure-testable and dark by default.
+
+## 4. Reversibility
+
+Fully reversible. The whole feature is dark behind
+`multiMachine.stateSync.relationships` (default `enabled:false`). Disabling it is a
+strict no-op (no emission, no union read, no peer view). A peer's contribution rolls
+back via the existing RollbackUnmerge (`/state/quarantine`). No destructive operation is
+introduced in THIS slice: the emit funnel only adds a best-effort `put`/tombstone
+AFTER the durable local write; it never blocks or alters a local mutation, and a
+throwing emitter cannot break a local write (proven). The destructive-local-delete-on-
+receive (mandate-gated) is explicitly OUT of this slice.
+
+## 5. Migration / fleet-rollout surface
+
+- **Config:** `multiMachine.stateSync.relationships { enabled:false, dryRun:true }` added
+  to `ConfigDefaults` — `applyDefaults` add-missing backfills existing agents (Migration
+  Parity; no bespoke migrateConfig block needed). Classified in `DARK_GATE_EXCLUSIONS`
+  (optional-integration); the hand-authored dark-gate line-map was recomputed via the
+  attributor (regeneration forbidden) and the new `relationships.enabled` path + the +14
+  shift below it hand-edited.
+- **CLAUDE.md awareness:** the existing "One Memory (replicated stores)" section gains a
+  WS2.3 relationships-consumer line (WITH the at-rest PII honesty note) in BOTH
+  `generateClaudeMd` (new agents) and `migrateClaudeMd` (existing agents, via an
+  idempotent content-sniff splicer guarded by the unique "Relationships are the FIRST PII
+  store" marker). The framework shadow markers already cover the section (both `**` and
+  `### ` variants), so Codex/Gemini parity is inherited.
+- **Dual-registry CI ratchet:** the `ReplicatedRecordEnvelope.test.ts` coupling test +
+  the new wiring test assert `relationship-record` is coupled in both registries — a
+  future edit removing it from `JOURNAL_KINDS` would RED.
+
+## Adversarial review (5 lenses — PII-grade)
+
+Total findings: **3 real, 3 folded** (0 deferred).
+
+1. **Injection / type-clamp completeness** — FINDING (folded): the `firstInteraction`/
+   `lastInteraction` fields render UNSANITIZED in `getContextForPerson`, so a foreign
+   record could smuggle markup through them. Folded: the schema validates them as
+   ISO-8601-only (rejecting any string with `<`/`>`/`"` or a non-date), and the foreign
+   render path additionally escapes every field inside the untrusted-data envelope —
+   two independent defenses. Proven by `injection-neutralized-firstInteraction` +
+   `schema-type-clamp`.
+2. **Disclosure minimization** — FINDING (folded): the local UUID `id` must never leave
+   the machine. Folded: it is absent from `RELATIONSHIP_STORE_KNOWN_FIELDS` and the
+   projection builder never emits it; the `disclosure-minimization` test asserts every
+   emitted field is in the allowlist and `id` is absent.
+3. **Tombstone / erasure correctness** — FINDING (folded): `mergeRelationships` could
+   emit a tombstone that suppresses the survivor (the merged channels are now part of
+   the survivor). Folded: the tombstone keys on the merged's OLD standalone channel set,
+   whose recordKey is DISTINCT from the survivor's consolidated recordKey — proven by
+   the merge test (distinct keys) and the union test (`erasure-reaches-offline-peer`,
+   tombstone-wins-over-stale-value). No dangling tombstone, no loop.
+4. **recordKey identity derivation** — no real finding: same channel set → same key
+   (order-independent); disjoint sets → different keys; the same person with different
+   UUIDs → same key. Collision requires the EXACT same full channel set, which is the
+   manager's own definition of "the same person". Proven by the recordKey tests.
+5. **Flag-coherence PII-leak prevention** — no real finding: emission is gated on
+   `isStoreEmissionEnabled` + the per-peer `shouldEmitToPeer` advertise check (foundation
+   machinery, REQ-M5); a dark agent never emits, and a peer that does not advertise
+   `relationships` is never a forward target. The advert self-reports from registry
+   presence + the enabled flag. (Live serve/pull transport is the later rollout stage,
+   gated by the same flag; THIS slice keeps PII strictly local until the operator flips
+   the flag.)
+
+## Verdict
+
+Safe to ship dark. Zero behavior change — and zero PII egress — for any agent until an
+operator flips `multiMachine.stateSync.relationships.enabled`. The channel-set recordKey,
+the type-clamped discriminated-union schema, the disclosure-minimized projection, the
+channel-keyed tombstone, and the untrusted-data render envelope are the load-bearing PII
+correctness properties and are each proven by a named test.