virtual-context 0.4.4__tar.gz → 0.5__tar.gz

{virtual_context-0.4.4 → virtual_context-0.5}/.gitignore

@@ -60,6 +60,7 @@ extraneous/paper/*.png
 *.json
 !pyproject.toml
 !package.json
+!tests/fixtures/*.json
 *.autopsy.md
 
 # OS
@@ -68,3 +69,7 @@ extraneous/paper/*.png
 
 # Riptide artifacts (cloud-synced)
 .humanlayer/tasks/
+
+# Unrelated local content kept in the working tree, never packaged
+discord-native-poker/
+discord-native-poker.zip

virtual_context-0.5/OVERNIGHT_ASSUMPTIONS.md

@@ -0,0 +1,92 @@
+# Overnight assumptions — cross-channel-mirror engine implementation
+
+Branch: `feature/cross-channel-mirror` (off `fix/vcattach-redis-marker-write`).
+
+Decisions made without an explicit spec/plan directive. Each entry names the call and the rationale so the reviewer can override quickly.
+
+## 1. Helper-module placement
+
+Per spec §2.6 the plan picked `virtual_context/core/protected_window.py`. I added the helpers there as specified (`_merge_protected_window`, `_stamp_canonical_turn_ids`, `_last_already_canonical_turn_number`).
+
+Within the merge helper, the `mode == "off"` branch returns `list(payload_history)` (a shallow copy) rather than the input by reference. Mirrors the "side-effect-free" rule from spec §2 even on the no-op path.
+
+## 2. `extract_ingestible_messages` mode parameter — wrapper kwarg
+
+`virtual_context/proxy/helpers.py:_extract_ingestible_messages` gained a kw-only `mode: str = "ingest"` that threads through to the underlying `extract_ingestible_messages`. Existing call sites in `proxy/server.py` (5 sites) and `proxy/state.py` (3 sites) do not pass `mode` and therefore inherit the default. No call-site changes needed.
+
+The `proxy/formats.py:514` internal self-call now explicitly passes `mode="ingest"` for clarity even though it's the default.
+
+## 3. `_decision` helper in `handle_prepare_payload`
+
+The 11 `PhaseDecision(phase=..., started_tagger=...)` return sites inside `handle_prepare_payload` were converted to a local helper `_decision(*, phase, started_tagger)` that closes over the local `canonical_ingest_rows` tuple. Closure uses Python's standard late-binding so the helper reads the current value at call time (after the `ingest_batch` block reassigns it). The original return statements at lines 1230-1445 became `return _decision(...)` calls.
+
+The `_decision` body uses an intermediate variable (`_decision_result = PhaseDecision(...)`) so the `return PhaseDecision(...)` text shape only exists once in the helper itself, which let `Edit(replace_all=True)` migrate the 11 return sites by text match without touching the helper body.
+
+## 4. Stamping placement in `proxy/server.py`
+
+Per plan §2 Step 7, stamping fires in the ACTIVE-path block at `proxy/server.py:1125-1131`, immediately before the active-tail user message is appended to `state.conversation_history`. Gated on `protected_window_db_source == "merge"` and `_phase_decision.canonical_ingest_rows` being non-empty. Wrapped in a defensive try/except so a stamping failure degrades to Tier 3 fall-through rather than failing the request.
+
+Alignment rule (suffix-tail drop on active-tail user) is implemented by calling `_extract_ingestible_messages(body)` + `state._completed_history_messages(...)` and dropping `len(extracted) - len(completed)` rows from the suffix.
+
+## 5. RetrievalAssembler reads `_is_merge_participant` via `getattr(self, ...)`
+
+The Tier 1 cached bool lives on the engine. Rather than add a constructor parameter to `RetrievalAssembler.__init__` (which would force every existing test fixture and `__new__` site to pass it), the engine assigns the bool onto the assembler instance after construction:
+
+```python
+self._retrieval._is_merge_participant = self._is_merge_participant
+```
+
+The assembler's gate code reads via `getattr(self, "_is_merge_participant", False)`, so tests that bypass engine construction via `RetrievalAssembler.__new__` (`engine.py:2366`) still work — they default to non-participant.
+
+## 6. Tier 2 INT coercion shape
+
+`int(redis_last_turn) == int(anchor_turn)` wrapped in `try/except (TypeError, ValueError)`. A coercion failure (e.g. the marker arrives as a dict instead of an int-coercible scalar) routes to `tier2_legacy_fallthrough` rather than raising. Tests cover both the string-coercible path (`"7"` → 7) and the non-coercible path (dict).
+
+## 7. Engine `__init__` Tier 1 call does NOT catch broad Exception
+
+Per plan §2 Step 8 explicit guidance: silently disabling the feature on a store-wrapper failure / missing allowlist / transient DB outage would hide mirror loss as stale-False. Only Tier 0 `off` short-circuits the call; merge mode propagates any exception from `has_any_alias`. This intentionally fails engine construction on misconfigured deploys.
+
+## 8. Tier 1 propagation across re-init paths
+
+Engine `__init__` sets `_is_merge_participant` once (line 220 area). The post-resurrect re-init path at engine.py:880+ keeps the same `RetrievalAssembler` instance (lines 941-946 mutate fields in place rather than re-constructing), so the assembler's `_is_merge_participant` attribute survives the re-init unchanged. No re-propagation is needed.
+
+The `RetrievalAssembler.__new__` fallback at engine.py:2366 (used by `filter_history` for test stubs) does not exercise `on_message_inbound`, so the gate code is never reached on that path; no Tier 1 propagation is needed there either.
+
+## 9. Observability: dedicated `PROTECTED_WINDOW_GATE` log line
+
+Per plan §2 Step 10 + §5.8, the gate observability is rendered INLINE as key=value substrings in the message text (cloud's stdlib-logging formatter drops `extra=`). I added a dedicated `logger.info("PROTECTED_WINDOW_GATE conv=%s ...", ...)` log line right after the existing INBOUND_BREAKDOWN block in `on_message_inbound`. The new line is gated on `_gate_outcome is not None`, so Tier 0 `off` emits nothing.
+
+The new line is NOT bound by the existing 500ms INBOUND_BREAKDOWN threshold — it fires on every `on_message_inbound` that runs in merge mode, which is the spec §8 observability contract.
+
+`extra=` is included for in-process consumers (tests using `caplog.records[i].my_field` attribute access) but the message text carries the values for grep-based consumption.
+
+## 10. `_gate_budget_evictions` reports 0 in current implementation
+
+The merge helper does NOT evict; the downstream context-builder eviction happens later and isn't currently surfaced via a hook to the gate path. The field is emitted as `0` for now; a future change can wire a real counter from the downstream context-builder if metrics analysis surfaces a need.
+
+## 11. Test files use the existing engine_factory pattern but DO NOT exercise filesystem backend for Tier 3
+
+`tests/test_engine_init_tier1.py` constructs full engines but seeds aliases via the SQLite path (the only backend that hosts canonical_turns). The filesystem backend's `has_any_alias` is tested at the unit level via `test_storage_protocols_cross_channel_mirror.py` (via SimpleNamespace stub) and is not exercised end-to-end on the engine path.
+
+## 12. Postgres tests skip when `VC_TEST_POSTGRES_URL` is not set
+
+`tests/test_postgres_mirror_store_methods.py` mirrors the always-on SQLite suite and uses `pytestmark = pytest.mark.skipif(not PG_URL, reason="VC_TEST_POSTGRES_URL not set")`. The CI environment is expected to set this when Postgres is wired; local dev skips cleanly.
+
+## 13. NO code changes to compaction / tagging / fact-curator / vc_* tools
+
+Plan §7 non-goals. Verified by grep — no edits to those modules.
+
+## 14. NO new indexes on conversation_aliases or canonical_turns
+
+Plan §1.2 + §7 non-goals. Both Tier 1 OR legs are already indexed (PK on `alias_id` + `idx_conversation_aliases_target_id`). Tier 3 read uses `idx_canonical_turns_conv_order` on `(conversation_id, sort_key)`.
+
+## 15. NO deploy, NO release, NO merge to main
+
+User explicitly revoked deploy authority tonight. Per spawn-prompt rule, release-class actions are user-direct only. Branch is pushed to `origin/feature/cross-channel-mirror` and stays there. Next step is for the user to review the diff and decide whether to merge `fix/vcattach-redis-marker-write` to main first, then this branch, OR fold both at once OR roll back.
+
+## Open follow-ups (NOT done tonight — explicit non-action)
+
+- Postgres EXPLAIN-plan smoke: I asserted catalog-level index presence in tests rather than `EXPLAIN ANALYZE` plan-node names (per plan §4 risk assessment — query planner choices vary on small test tables).
+- End-to-end harness coverage: the `tests/fixtures/target_baseline.sql` already-shipped baseline can be used by the cloud-side harness to exercise the cross-channel-mirror end-to-end. The cloud-side test authoring is owned by cloud per spec §8.
+- Spec drift on YAML key (`assembler:` vs actual `assembly:`): plan §3.1 documents this as a follow-up amendment to the engine spec + cloud spec. Implementation uses the correct `assembly:` key.
+- `cross-channel-mirror-engine-spec.md` line 261 `cloud-spec §1.1` reference: that section in cloud-spec is currently named `1.4` and `5.5` per plan §3.1. Spec amendment pending.

{virtual_context-0.4.4 → virtual_context-0.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: virtual-context
-Version: 0.4.4
+Version: 0.5
 Summary: OS-style virtual memory for LLM session context management
 Project-URL: Homepage, https://virtual-context.com
 Project-URL: Repository, https://github.com/virtual-context/virtual-context
@@ -51,7 +51,7 @@ Description-Content-Type: text/markdown
 <!-- [![Python](https://img.shields.io/pypi/pyversions/virtual-context.svg)](https://pypi.org/project/virtual-context/) -->
 <!-- [![Downloads](https://img.shields.io/pypi/dm/virtual-context.svg)](https://pypistats.org/packages/virtual-context) -->
 <!-- [![License](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](https://github.com/yursilkidwai/virtual-context/blob/main/LICENSE) -->
-[![Discord](https://img.shields.io/badge/Discord-Chat%20with%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/YxDHKEZz)
+[![Discord](https://img.shields.io/badge/Discord-Chat%20with%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/kGJva2D8Ej)
 [![Twitter](https://img.shields.io/badge/Twitter-Follow-1DA1F2?logo=x&logoColor=white)](https://x.com/virtualctx)
 
 <p align="center">

{virtual_context-0.4.4 → virtual_context-0.5}/README.md

@@ -2,7 +2,7 @@
 <!-- [![Python](https://img.shields.io/pypi/pyversions/virtual-context.svg)](https://pypi.org/project/virtual-context/) -->
 <!-- [![Downloads](https://img.shields.io/pypi/dm/virtual-context.svg)](https://pypistats.org/packages/virtual-context) -->
 <!-- [![License](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](https://github.com/yursilkidwai/virtual-context/blob/main/LICENSE) -->
-[![Discord](https://img.shields.io/badge/Discord-Chat%20with%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/YxDHKEZz)
+[![Discord](https://img.shields.io/badge/Discord-Chat%20with%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/kGJva2D8Ej)
 [![Twitter](https://img.shields.io/badge/Twitter-Follow-1DA1F2?logo=x&logoColor=white)](https://x.com/virtualctx)
 
 <p align="center">

{virtual_context-0.4.4 → virtual_context-0.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "virtual-context"
-version = "0.4.4"
+version = "0.5"
 description = "OS-style virtual memory for LLM session context management"
 readme = "README.md"
 license = "AGPL-3.0-or-later"
@@ -77,6 +77,7 @@ exclude = [
     ".github/",
     ".gitignore",
     ".worktrees/",
+    "tests/fixtures/",
 ]
 artifacts = [
     "virtual_context/proxy/static/*",

{virtual_context-0.4.4 → virtual_context-0.5}/tests/REGRESSION_MAP.md

@@ -374,6 +374,190 @@ Use `pytest -m regression` to run all regression tests.
   - `test_conversation_scoping.py::TestEndToEndScoping::test_find_quote_scoped_no_cross_conversation_leaks`
   - `test_conversation_scoping.py::TestEndToEndScoping::test_alias_ride_along_scoped`
 
+### BUG-036 — Sort-key gap exhaustion collides with boundary rows
+
+- **Symptom**: Prepare-payload ingest aborts with `UNIQUE (conversation_id, sort_key)` violation (Postgres UniqueViolation / SQLite IntegrityError) on mid-insertion into a conversation whose insertion-point rows sit ≤0.002 apart; large mid-inserts also silently mis-order rows past the right boundary.
+- **Root cause**: `IngestReconciler._allocate_sort_keys` clamped its step to 0.001 when the bounded gap was too tight, letting allocated keys land ON or PAST `right_key`. Self-priming: the first clamped allocation writes 0.001-spaced rows, after which every insertion between them collides deterministically.
+- **Fix**: Bounded allocation signals exhaustion (returns `None`) instead of clamping, with a float-precision strict-interior guard. The reconciler then opens the gap via `shift_canonical_turn_sort_keys` (single UPDATE whose delta exceeds the shifted range's spread — transient-collision-safe in any row visit order) and re-allocates; per-row descending fallback for stores without the bulk helper.
+- **Tests**:
+  - `test_ingest_sort_key_rebalance.py::TestAllocatorBoundedGap` (exhaustion signaling)
+  - `test_ingest_sort_key_rebalance.py::TestMidInsertRebalance` (end-to-end rebalance, self-priming loop, rows_touched consistency, per-row fallback)
+  - `test_ingest_sort_key_rebalance.py::TestShiftHelperSQLite`
+  - `test_sort_key_shift_postgres.py::TestShiftHelperPostgres` / `TestMidInsertRebalancePostgres`
+
+### BUG-038 — Strict tagging aborts on concurrently-tagged rows
+
+- **Symptom**: `RuntimeError: strict canonical tagging expected at least N covered ingestible entries, found M across K rows` on catch-up ingestion of a multi-worker conversation; the whole batch falls through to the row-based DB sweep, losing payload-context tagging for the still-untagged rows.
+- **Root cause**: The strict precondition in `ingest_history` required every payload entry to map to an UNTAGGED canonical row. Rows legitimately tagged between the prepare and the follow-up pass — by another worker's row sweep or a prior pass over an overlapping payload — broke the count.
+- **Fix**: The precondition counts coverage over the conversation's row tail INCLUDING tagged rows. The pair walker gains a hydrate fast-path: pairs whose backing rows are all tagged (matched by per-message `turn_hash`) get their TurnTagIndex entries from the stored row tags, consuming the strict cursor without invoking the tag generator or rewriting rows. Half-tagged pairs fall through to the normal tagger (idempotent). Supporting fix: the canonical-turn full-row loaders now SELECT `covered_ingestible_entries` so legacy combined rows (coverage 2) count correctly.
+- **Tests**: `test_strict_tagging_tagged_rows.py` — prod-signature repro, hydration tag fidelity, zero-tagger-call full hydration, untagged-tail still tagged, half-tagged fall-through, missing-rows and stale-epoch invariants preserved.
+
+### BUG-048 — Reconciliation loads every column of every stored turn, including the text it never reads
+
+- **Symptom**: Ingest cost scales with stored conversation size and is independent of request size, and the per-row cost grows with row width rather than staying flat. A no-content projection of the same rows measures flat at 4.2-5.6 us/row across a 7x range in conversation size while the full row goes 22.9 to 95.2 us/row over the same range.
+- **Root cause**: `IngestReconciler` loaded stored history through `get_all_canonical_turns`, which selects every column. Reconciliation keys on `turn_hash`, `sort_key` and identity/provenance columns, and never reads a stored row's text; the only thing it asks of the content is whether a row carries user text at all, as a role gate before taking speaker attribution. The content columns are the widest part of a row and are stored out of line, so on Postgres they dominate a load that never uses them.
+- **Fix**: New `CanonicalTurnReconcileRow` and `get_canonical_turn_reconcile_rows` on both backends, selecting only the reconciliation columns plus `has_user_content` / `has_assistant_content` flags. All three reconciler loads go through `_load_reconcile_rows`, which falls back to the full load when a backend returns None; None is distinguished from an empty list because reading "cannot project" as "no rows" would reconcile every payload against no history. It is deliberately a separate type rather than a `CanonicalTurnRow` with empty text, so a projected row reaching a write path raises instead of blanking stored content, which is what the sort-key rebalance fallback surfaced. `exact_resend` now returns the prepared rows carrying the stored ordinal rather than the stored rows themselves, so `CanonicalIngestResult.rows` is one type on every path.
+- **Tests**:
+  - `test_ingest_projected_rows.py::test_presence_flag_agrees_with_python_strip` (the flag and `(value or "").strip()` must never disagree, including whitespace-only content; the default single-argument TRIM strips spaces only and reports a newline-only row as carrying content)
+  - `test_ingest_projected_rows.py::test_attribution_still_persists_through_the_projection` (a late-derived sender still reaches storage through the role gate)
+  - `test_ingest_projected_rows.py::test_projection_preserves_every_field_the_enrichment_merge_reads` (each merge-read column carries a distinguishing value, so a blanked projection is caught)
+  - `test_ingest_projected_rows.py::test_result_rows_never_carry_a_projected_row` (across exact_resend, append and interior_overlap)
+  - `test_ingest_projected_rows.py::test_exact_resend_rows_carry_the_stored_ordinal`
+  - `test_ingest_projected_rows.py::test_projected_row_cannot_be_written_back`
+  - `test_ingest_projected_rows.py::test_unsupported_backend_falls_back_to_the_full_load`
+
+### BUG-047 — Anchor refresh rewrites the whole conversation's anchor table on every ingest
+
+- **Symptom**: Ingest cost grows linearly with stored conversation size and is independent of what the caller sent. A 15-message request against a 9,173-row conversation costs the same as a 490 KB one. On the largest production conversation the prepare path exceeded the client's timeout often enough that a quarter of prepares were abandoned before a response was sent.
+- **Root cause**: `_refresh_persisted_anchors` rebuilt the complete anchor set on every call and persisted it through `replace_canonical_turn_anchors`, which is a `DELETE` of every anchor row for the conversation followed by a re-INSERT of the entire rebuilt set. The set holds one row per window start per window size (3, 4, 5), so roughly 3N rows are deleted and 3N re-inserted per ingest. Appending a turn creates exactly one new window start per window size and invalidates none, so nearly the whole rewrite was writing back byte-identical rows. Measured on a 9,200-row conversation: 27,597 anchor rows rewritten, of which 6 were new and 0 removed. Both ingest fast paths (`exact_resend`, `tail_append`) paid it too, and on `exact_resend` the anchor set is provably unchanged, making the entire rewrite dead work.
+- **Fix**: `_build_anchor_rows` extracted so the rebuild and the delta derive their sets with one ruler. `_refresh_persisted_anchors` accepts the row sequence persisted before the ingest, derives the required anchor set for both sequences in memory, and writes only the difference through the new `apply_canonical_turn_anchor_delta` store surface, which deletes and inserts exact `(window_size, anchor_hash, start_turn_id)` triples scoped to the conversation. Because the delta is derived from the caller's prior sequence rather than from a read of stored anchors, it is applied only when `count_canonical_turn_anchors` agrees with the derived prior set; any disagreement (anchors never built, a torn write, a concurrent writer) falls back to the full rebuild, preserving the self-healing property the unconditional rebuild provided. Callers that cannot name the prior sequence keep the rebuild.
+- **Tests**:
+  - `test_ingest_anchor_incremental.py::test_incremental_delta_matches_full_rebuild` (the identity property across append, interior insert, prefix insert, in-place modification, interior removal, tail shrink, shrink below window size, and wholesale replacement)
+  - `test_ingest_anchor_incremental.py::test_incremental_result_is_order_independent` (both temporal orderings converge on the canonical rebuild)
+  - `test_ingest_anchor_incremental.py::test_append_writes_bounded_anchor_rows_not_whole_conversation` (an append writes 3 anchors regardless of conversation length; a full-conversation rewrite fails this)
+  - `test_ingest_anchor_incremental.py::test_unchanged_sequence_writes_nothing` (a resend that changes no row issues no anchor write)
+  - `test_ingest_anchor_incremental.py::test_diverged_store_falls_back_to_full_rebuild`
+  - `test_ingest_anchor_incremental.py::test_anchor_hashes_still_resolve_windows_after_delta` (post-delta digests still map to their start row, and superseded digests do not survive)
+  - `test_ingest_anchor_incremental.py::test_sqlite_delta_write_matches_rebuild_write` (the SQL delta leaves the same table state as the wholesale replace)
+### BUG-044 — Compaction offset splits a recovered cross-channel turn
+
+- **Symptom**: A temporary reply preference authored in one Discord guild
+  channel is acknowledged there but ignored in another channel. The outbound
+  recent-conversation block contains only the assistant acknowledgement, not
+  the authoritative user instruction.
+- **Root cause**: Tier 3 inserts `source=db_recent` rows into channel-local
+  payload history before `history_offset()` is applied. The payload-owned
+  compaction offset is then used as a raw index into the merged list, so it can
+  consume the user half of a recovered logical group while leaving the
+  assistant half.
+- **Fix**: Build the uncompacted view with a source-aware, active-tail-safe
+  slice: consume the offset from payload-owned rows only, never from DB-recent
+  rows or the trailing unstamped active-user block.
+- **Tests**:
+  - `test_protected_window_gate.py::test_tier3_db_pair_survives_payload_compaction_offset`
+
+### BUG-045 — Payload dedup suppresses the canonical user before offset
+
+- **Symptom**: A guild-wide temporary reply preference is acknowledged in its
+  source Discord channel but ignored in another channel. Canonical storage
+  contains the correctly attributed user instruction and assistant
+  acknowledgement, yet the outbound `<recent-conversation>` contains only the
+  assistant acknowledgement.
+- **Root cause**: Tier 3 dedup ran before the channel-local payload watermark.
+  The payload user suppressed its canonical duplicate by
+  `source_message_id`, then `history_offset()` removed the payload copy. The
+  unkeyed canonical assistant half survived alone. Separately, the Tier 3
+  store query limited physical rows and could split a logical group at the
+  oldest window boundary.
+- **Fix**: Slice payload-owned history before canonical merge, force Tier 3
+  when a nonzero payload offset makes a Tier-2 equality skip unsafe, and cache
+  that exact merged view for paging reassembly. Dedup uses adjacency-scoped
+  logical groups so reused historical group numbers cannot suppress unrelated
+  turns; incomplete payload fragments are replaced by the complete canonical
+  group, then recovered groups are interleaved ahead of newer stamped payload
+  turns. Legacy adjacency is never accepted as channel/audience provenance
+  proof. A failed/empty canonical read preserves the unsliced payload. SQLite
+  and Postgres select the newest N logical groups and return every physical
+  row in each.
+- **Tests**:
+  - `test_protected_window_gate.py::test_tier3_payload_duplicate_cannot_suppress_db_user_before_offset`
+  - `test_protected_window_gate.py::test_tier2_equal_with_payload_offset_forces_canonical_replacement`
+  - `test_protected_window_gate.py::test_tier3_read_failure_preserves_unsliced_payload`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_source_match_suppresses_entire_split_db_group`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_replaces_incomplete_payload_fragment_with_canonical_pair`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_replacement_keeps_canonical_group_before_newer_payload`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_complete_payload_pair_can_span_tool_scaffolding`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_dedup_does_not_conflate_reused_group_numbers`
+  - `test_retrieval_assembler_protected_window_merge.py::test_merge_legacy_adjacency_does_not_inherit_channel_provenance`
+  - `test_sqlite_mirror_store_methods.py::test_get_recent_limit_preserves_complete_split_groups`
+  - `test_sqlite_mirror_store_methods.py::test_get_recent_limit_does_not_conflate_reused_group_number`
+  - `test_postgres_mirror_store_methods.py::test_get_recent_limit_preserves_complete_split_groups`
+  - `test_postgres_mirror_store_methods.py::test_get_recent_limit_does_not_conflate_reused_group_number`
+  - `test_recent_conversation_assembly.py::test_budget_does_not_conflate_reused_raw_group_numbers`
+  - `test_recent_conversation_assembly.py::test_budget_legacy_fallback_evicts_only_leading_contiguous_group`
+
+### BUG-046 — Retained peer-channel history suppresses the canonical copy
+
+- **Symptom**: The complete newest preference pair is correct in canonical
+  storage but absent from the next channel's `<recent-conversation>`. Older
+  groups render normally, and the preference continues to work only in the
+  channel where it was authored.
+- **Root cause**: A unified guild reuses one in-memory engine history across
+  channels, while Discord sends channel-local model payloads. Tier 3 treated
+  any matching retained engine row as proof that the current payload already
+  carried the canonical turn, so the immediately preceding peer-channel group
+  was suppressed even though it was not model-visible. An active-only ingest
+  result could also suffix-stamp that current row's identity onto older
+  completed history when the active-tail drop equaled the entire result.
+- **Fix**: When a request has a proved origin channel, only retained rows from
+  that exact channel may suppress a canonical copy; production nested channel
+  metadata is read through `get_origin_channel`, with the DB-recent top-level
+  shape as fallback. Active-tail ingest rows are removed with an empty-safe
+  helper before completed-history stamping. Requests without a proved channel
+  retain legacy dedup behavior.
+- **Tests**:
+  - `test_protected_window_gate.py::test_other_channel_engine_history_cannot_suppress_canonical_recent_pair`
+  - `test_protected_window_gate.py::test_same_channel_payload_twin_still_suppresses_canonical_copy`
+  - `test_protected_window_gate.py::test_same_channel_active_tail_race_still_suppresses_db_user`
+  - `test_canonical_turn_id_stamping.py::test_drop_active_tail_equal_to_all_rows_returns_empty`
+  - `test_canonical_turn_id_stamping.py::test_drop_active_tail_keeps_completed_prefix`
+  - `test_canonical_turn_id_stamping.py::test_drop_active_tail_zero_is_identity_view`
+
+### BUG-043 — RRF's missing-signal penalty buries embedding-only candidates below the fused top-K
+
+- **Symptom**: On analog queries with no keyword overlap, the tag the embedding signal surfaces most strongly lands far outside the fused top-K (embedding candidates observed at fused rank 15-33 while the selection cut is 10), because RRF penalizes every candidate absent from the idf/bm25 signals. The context-augmented embedding signal (BUG-042) surfaces the right tag but fusion then discards it.
+- **Root cause**: `score_candidates` fuses three ranked signals with a missing-signal penalty rank; a strong embedding-only candidate carries penalty ranks for idf and bm25 and cannot reach the top-K on the embedding weight alone.
+- **Fix**: New `retrieval.scoring.embedding_reserved_seats` (default 0 = byte-identical legacy). After fusion/dampening/boost and before the top-K cut, `apply_embedding_reserved_seats` seats the top-N embedding-signal candidates (raw pre-gravity embedding order) that are outside the fused top-K (K = strategy `max_results`) by raising their fused scores into the gap between the last surviving entry and the first displaced one, so the retriever's score-descending cut yields surviving + reserved. Only the seated tags' scores change; a degenerate boundary tie nudges the displaced entries down. Composes with BUG-042.
+- **Tests**:
+  - `test_embedding_reserved_seats.py::TestReservedSeatMath` (seating parity vs the rule on a synthetic fixture and an anonymized real prod fused/embedding dump; lowest-of-top-K displaced)
+  - `test_embedding_reserved_seats.py::TestLegacyByteIdentical` (N=0 no mutation, everything-already-fits, embedding-already-in-top-K, defaults)
+  - `test_embedding_reserved_seats.py::TestNoGoldDisplacement` (buried gold seated without evicting gold already in top-K; seated score lands between survivor and displaced)
+  - `test_embedding_reserved_seats.py::TestConfigPlumbing`
+
+### BUG-042 — Under-specified analog queries miss the relevant tag on the bare embedding signal
+
+- **Symptom**: The embedding retrieval signal scores only the bare inbound message against tag-summary vectors, so an under-specified query whose surface words name neither the gold tag nor the summary's distinctive nouns ("what should I get her for her birthday") fails to surface the relevant tag even when the elided topic sits in the immediately preceding turns.
+- **Root cause**: `compute_embedding_candidates` had one query vector — the bare message embedding produced by the inbound embedding tagger, which discards the conversational context it receives. Recent-turn context that would disambiguate the query was never folded into signal 3.
+- **Fix**: New `retrieval.scoring.embedding_context_turns` (default 0 = byte-identical legacy) blends the last N conversational turns + the current message into a second query embedding on the same encoder. `embedding_context_guard` (default true) takes the per-tag MAX of the bare and context similarities before threshold/ranking, so a tag can only ever rank at least as well as it would from the bare query alone — irrelevant recent context cannot demote a relevant tag below its bare rank. Guard false scores the context vector alone (plain concat, experimentation).
+- **Tests**:
+  - `test_embedding_context_guard.py::TestGuardMathParity` (per-tag max ranking matches the reference guard math; guard sim >= bare sim per tag)
+  - `test_embedding_context_guard.py::TestLegacyByteIdentical` (N=0 / no-context path identical to legacy; defaults are 0/true)
+  - `test_embedding_context_guard.py::TestCraterRegression` (irrelevant-context crater repaired: plain concat demotes the gold tag, guard recovers it near the bare rank)
+  - `test_embedding_context_guard.py::TestConfigPlumbing` / `TestRetrieverContextEmbedding` (YAML parse + retriever concat construction)
+
+### BUG-041 — Compaction materializes tag summaries for only the greedy cover
+
+- **Symptom**: Every compaction leaves segment tags with no `tag_summaries` row (~69/day on a live multi-tag conversation); those tags are invisible to the context-hint topic list, absent from the broad/recall-all summary floor, and missing from tag-summary-embedding scoring. Rows materialized by an external repair sweep go permanently stale because later compactions keep skipping the tags. First observed May 30 (tags present in `segment_tags`, absent from `tag_summaries`).
+- **Root cause**: By-design minimality that contradicted the system's own read and repair contracts: `_build_tag_summaries` intersected the greedy set-cover with the compacted segments' tags (plus a primary-tag guarantee), structurally omitting every non-primary secondary tag outside the cover. The backfill repair and the read paths both assume every non-`_general` segment tag has a summary.
+- **Fix**: Compaction materializes a summary for every non-`_general` tag carried by the just-compacted segments; the primary-tag guarantee is unchanged. The pre-existing staleness check inside `compact_tag_summaries` bounds the LLM cost (fresh summaries skip), and secondary tags now also REFRESH on later compactions instead of going stale.
+- **Tests**:
+  - `test_tag_summary_materialization.py` — no gap after an organic compaction, `_general` exclusion, report coverage, secondary-tag refresh eligibility on later compactions
+  - `test_tag_summary_backfill.py::test_compaction_tag_summaries_populated_when_in_memory_index_empty` (pin updated to the wide contract)
+
+### BUG-040 — Assistant-half ingest duplicates the prepared user row
+
+- **Symptom**: Three linked production symptoms on prepare-then-ingest REST conversations: (1) strict tagging fails with "could not map payload messages to existing rows for logical turn N" and turns fall through to the context-free row sweep (observed as a jump in `_general` primary tags on live traffic), (2) canonical rows contain duplicated user halves and mid-inserted copies of already-present content, (3) sort-key gaps at the insertion point halve with every prepare, priming the BUG-036 gap-exhaustion collisions.
+- **Root cause**: `ingest_single` (the completed-pair persist path) prepares a `[user, assistant]` row pair, but the user half is normally already the conversation's LAST row, persisted by the preceding payload prepare. A 2-row incoming fragment has no ≥3-row anchor window and short-overlap matching is disallowed on this path, so alignment failed and `no_overlap_append` duplicated the user row; the duplicate scrambled every subsequent payload alignment.
+- **Fix**: Before falling through to full alignment, `ingest_single` matches the pair's user hash against the tail row; on a match it mirrors the existing row's identity (no rewrite) and appends only the assistant row (`tail_append`). Resend dedup and genuinely-new-pair append behavior are unchanged.
+- **Tests**: `test_ingest_single_tail_pair.py` — no duplication, multi-turn row cleanliness, strict tagging green across a full REST conversation, resend dedup preserved, unprepared-pair append preserved.
+
+### BUG-039 — engine_state saves fail on schema-bootstrapped Postgres
+
+- **Symptom**: `Failed to save engine state: column "flushed_prefix_messages" of relation "engine_state" does not exist` on every engine-state save; swallowed as a warning, so session-restore state silently never persists.
+- **Root cause**: The bundled schema created `engine_state` with five columns while `save_engine_state` on Postgres INSERTs seven (`flushed_prefix_messages`, `last_request_time`).
+- **Fix**: Both columns added to the bundled CREATE TABLE, plus idempotent `ADD COLUMN IF NOT EXISTS` migration in the schema bootstrap for tables created by earlier schemas.
+- **Tests**:
+  - `test_engine_state_schema_postgres.py::test_engine_state_table_has_save_path_columns`
+  - `test_engine_state_schema_postgres.py::test_save_engine_state_round_trips_on_fresh_schema`
+
+### BUG-037 — Schema bootstrap DDL races across workers
+
+- **Symptom**: Multi-worker startup logs `tuple concurrently updated` from `CREATE OR REPLACE FUNCTION` (and historically DuplicateObject from trigger DROP+CREATE pairs); the losing worker logs "merge_post_commit_pending bootstrap failed" and skips the rest of its guarded block.
+- **Root cause**: `PostgresStore._ensure_schema` ran all bootstrap DDL with no cross-worker serialization.
+- **Fix**: Session-scoped `pg_advisory_lock` held on a dedicated connection for the whole bootstrap (`_SCHEMA_BOOTSTRAP_LOCK_KEY`); session-scoped rather than xact-scoped because the bootstrap relies on per-statement autocommit for its try/except-guarded statements.
+- **Tests**:
+  - `test_schema_bootstrap_postgres.py::TestSchemaBootstrapSerialization::test_concurrent_bootstrap_no_ddl_race_warnings`
+
 ---
 
 ## By Test File
@@ -397,3 +581,19 @@ Use `pytest -m regression` to run all regression tests.
 | `test_verb_expansion.py` | BUG-032 |
 | `test_engine_integration.py` | BUG-034 |
 | `test_conversation_scoping.py` | BUG-035 |
+| `test_ingest_sort_key_rebalance.py` | BUG-036 |
+| `test_sort_key_shift_postgres.py` | BUG-036 |
+| `test_schema_bootstrap_postgres.py` | BUG-037 |
+| `test_strict_tagging_tagged_rows.py` | BUG-038 |
+| `test_engine_state_schema_postgres.py` | BUG-039 |
+| `test_ingest_single_tail_pair.py` | BUG-040 |
+| `test_tag_summary_materialization.py` | BUG-041 |
+| `test_embedding_context_guard.py` | BUG-042 |
+| `test_embedding_reserved_seats.py` | BUG-043 |
+| `test_ingest_anchor_incremental.py` | BUG-047 |
+| `test_ingest_projected_rows.py` | BUG-048 |
+| `test_protected_window_gate.py` | BUG-044, BUG-045, BUG-046 |
+| `test_canonical_turn_id_stamping.py` | BUG-046 |
+| `test_retrieval_assembler_protected_window_merge.py` | BUG-045 |
+| `test_sqlite_mirror_store_methods.py` | BUG-045 |
+| `test_postgres_mirror_store_methods.py` | BUG-045 |

{virtual_context-0.4.4 → virtual_context-0.5}/tests/conftest.py

@@ -2,7 +2,14 @@
 
 from __future__ import annotations
 
+import os
 import tempfile
+
+# Embedding models are pre-cached; without these flags each model
+# construction performs a network revision check against the hub CDN,
+# which can stall the suite for minutes under a degraded network.
+os.environ.setdefault("HF_HUB_OFFLINE", "1")
+os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")
 from datetime import datetime, timedelta, timezone
 from pathlib import Path
 

virtual_context-0.5/tests/fixtures/reserved_seats_diag_ctx.json

@@ -0,0 +1,72 @@
+{
+ "embedding_scores": {
+  "tag_001": 0.392342209815979,
+  "tag_002": 0.42588120698928833,
+  "tag_003": 0.39020001888275146,
+  "tag_007": 0.41312387585639954,
+  "tag_009": 0.3971983790397644,
+  "tag_011": 0.45882731676101685,
+  "tag_013": 0.451797753572464,
+  "tag_020": 0.41635698080062866,
+  "tag_027": 0.4214135408401489,
+  "tag_028": 0.49210599064826965,
+  "tag_030": 0.5612373352050781,
+  "tag_031": 0.4008348286151886,
+  "tag_033": 0.42422300577163696,
+  "tag_034": 0.40469276905059814,
+  "tag_035": 0.40652984380722046,
+  "tag_036": 0.4652409553527832,
+  "tag_038": 0.44060882925987244,
+  "tag_039": 0.4001418948173523,
+  "tag_040": 0.46100878715515137,
+  "tag_043": 0.4096267521381378
+ },
+ "fused_scores": {
+  "tag_000": 0.01095915225070193,
+  "tag_001": 0.006983992066864994,
+  "tag_002": 0.007404964129628102,
+  "tag_003": 0.006951535072382683,
+  "tag_004": 0.010115372115697107,
+  "tag_005": 0.00901243093922652,
+  "tag_006": 0.009519187695983276,
+  "tag_007": 0.007236790911213134,
+  "tag_008": 0.00864478388040299,
+  "tag_009": 0.007017292100165029,
+  "tag_010": 0.01057493093922652,
+  "tag_011": 0.007544889502762431,
+  "tag_012": 0.009255937432733013,
+  "tag_013": 0.007496812579685508,
+  "tag_014": 0.009905288082083663,
+  "tag_015": 0.009091544863277153,
+  "tag_016": 0.008576384427598612,
+  "tag_017": 0.008935270445399358,
+  "tag_018": 0.008714811891607471,
+  "tag_019": 0.010698938875734455,
+  "tag_020": 0.007277032359905287,
+  "tag_021": 0.008786527324768688,
+  "tag_022": 0.009341378307647571,
+  "tag_023": 0.008859991914836276,
+  "tag_024": 0.010826947068258777,
+  "tag_025": 0.00917268734948293,
+  "tag_026": 0.010454738631534212,
+  "tag_027": 0.007318440227400112,
+  "tag_028": 0.007698578027352595,
+  "tag_029": 0.00980468446035328,
+  "tag_030": 0.006919889502762431,
+  "tag_031": 0.007086556169429097,
+  "tag_032": 0.010225117506390699,
+  "tag_033": 0.007361065973350666,
+  "tag_034": 0.011606538077832818,
+  "tag_035": 0.007159615530159691,
+  "tag_036": 0.007645695954375333,
+  "tag_037": 0.009706875383670963,
+  "tag_038": 0.007450192533065461,
+  "tag_039": 0.007051468450130851,
+  "tag_040": 0.007594492677365605,
+  "tag_041": 0.009429097605893187,
+  "tag_042": 0.00961174600771967,
+  "tag_043": 0.007197667280540209,
+  "tag_044": 0.010338188514984095
+ },
+ "note": "Anonymized real prod fused/embedding dump (context-augmented analog query). Tag names opaqued; scores preserved verbatim for reserved-seat parity."
+}