hippo-memory 1.8.1 → 1.9.3

package/README.md

@@ -85,6 +85,33 @@ hippo recall "data pipeline issues" --budget 2000
 
 ---
 
+### What's new in v1.9.3
+
+- **Reranker review-tail patch.** Closes the three follow-ups raised on PR #25: `src/rerankers/llm.ts` now wires `AbortController` + `setTimeout` around the fetch (default 30 s, overridable via `HIPPO_LLM_RERANKER_TIMEOUT_MS`) so recall never hangs on a wedged endpoint; `src/rerankers/cross-encoder.ts` emits a single `console.warn` on first identity-fallback per process so silent fallback no longer masquerades as a working reranker; the orphan `RerankSignals` type (sole consumer retracted in v1.9.1) is removed at both the re-export and the definition.
+- **Version alignment.** `package.json` bumped 1.8.1 → 1.9.3. v1.9.0 / v1.9.1 / v1.9.2 were on-master research milestones never published to npm; v1.9.3 is the first published `1.9.x` release and carries the cumulative scope from F6 (rerankers) through F13 (chunk-per-turn) plus the F10 HARD RETRACTION.
+- **Mechanism cumulative-null status unaffected.** Per `docs/RETRACTION.md:94-113`. No `src/` change in this patch touches the dlPFC goal-stack mechanism. **This release does not re-assert the retracted −10pp magnitude.**
+
+### What's new in v1.9.2
+
+- **F13 chunk-per-turn LongMemEval R@5 = 86.8 on oracle (Gate-B PASS).** Plan F13 (`docs/evals/2026-05-12-r5-track6-chunk-per-turn-prereg.md`) addresses the structural pathology that limited every prior LongMemEval track (F8–F12): sessions in `data/longmemeval_oracle.json` are ~14k chars median (~3,500 tokens), but the embedders we can reach (MiniLM, BGE-base, multilingual-e5-large) cap at 512–514 tokens. Every prior track embedded only the first ~2 turns of each 12-turn session and truncated the rest. F13 replaces session-level embedding with turn-level embedding (10,866 turns over the 940 oracle sessions, max-pool by `session_id` at retrieval). Gate-A PASS (10,866 turns, all 940 sessions covered, 768-dim normalized). **Gate-B PASS:** F13 + F9 sub-agent rerank stack R@5 = 86.8 on `data/longmemeval_oracle.json` (threshold ≥ 83.2 = F11+F9 deployable best 78.2 + 5pp; margin 3.6). R@1 = 70.8, R@10 = 90.2, R@20 = 93.4.
+- **Roadmap target met (oracle split).** R@5 ≥ 85% was NON-binding per every prior prereg; observed 86.8 on `data/longmemeval_oracle.json` as of this release. Descriptive characterisation; not a re-assertion of any retracted magnitude.
+- **Split-mismatch with gbrain (unchanged).** `longmemeval_oracle` carries 3 sessions per haystack; gbrain v0.28.8's 97.60 figure is on `longmemeval_s_cleaned` (~40 sessions per haystack) with OpenAI `text-embedding-3-large@1536`. Both HF Hub and OpenAI API are host-blocked from this sandbox (verified 2026-05-12). F13's 86.8 is NOT directly comparable to gbrain's 97.60.
+- **F12 retracted.** Plan F12 (`docs/evals/2026-05-11-r5-track5-e5-large-top100-prereg.md`) vendored `intfloat/multilingual-e5-large` and widened the candidate pool to top-100. Gate-A PASS; Gate-B FAIL with best variant R@5 = 78.8 (threshold 83.2). HARD RETRACTION executed: `hippo_store2/` reverted to BGE-base; the `prefixFor` / `preferredBackend` dispatch helpers stay in `src/embeddings.ts` per the dispatch-shape carve-out (they return the legacy behaviour for non-e5 models).
+- **No `src/` changes in v1.9.2.** F13 is implemented as `benchmarks/longmemeval/chunk_per_turn_{embed,retrieve}.mjs` and reuses F11/F12's existing dispatch helpers. The cumulative-null status of the dlPFC goal-stack mechanism (`docs/RETRACTION.md:94-113`) is unaffected. **This release does not re-assert the retracted −10pp magnitude.**
+
+### What's new in v1.9.1
+
+- **F10 features-reranker retraction.** Plan F10 (`docs/plans/2026-05-11-r5-track3-richer-ingest.md`) tested whether populating entry-level signals via 19 Claude-sub-agent invocations would let the features reranker move R@5 above features-default + 5pp on LongMemEval. Observed: features-enriched R@5 = 59.2 vs features-default R@5 = 75.8 (same bge-base embedding model), a 21.6pp shortfall against the binding gate. Per the prereg's HARD RETRACTION clause, `src/rerankers/features.ts` + its test + its micro-fixture + its dispatcher case are removed in v1.9.1. The Track 2 cross-encoder and Track 3 LLM-rerank skeletons are preserved. **This release does not re-assert the retracted −10pp magnitude.** Per `docs/RETRACTION.md`.
+- **F11 embedding upgrade tested and documented (not shipped as default).** Plan F11 (`docs/plans/2026-05-11-r5-track4-embedding-upgrade.md`) swapped `Xenova/all-MiniLM-L6-v2` for `BAAI/bge-base-en-v1.5` (768-dim, CLS pooling). Gate-A PASS; Gate-B FAIL (R@5 = 77.0% vs threshold 81.8%). The `poolingFor` per-model dispatch in `src/embeddings.ts` and the `--model` flag in `scripts/fetch_embedding_model.mjs` ship; MiniLM remains the project default.
+- **Cross-track R@5 status (as of v1.9.1):** F8 hybrid tuning (MiniLM) 76.8, F9 v2 sub-agent LLM rerank (MiniLM) 78.0, F11 bge-base baseline 77.0, F11+F9 stack (BGE-base + sub-agent rerank) 78.2 — cross-track best at v1.9.1 — F10 features-enriched (retracted) 59.2. Roadmap target R@5 ≥ 85% was NOT MET at v1.9.1. NON-binding per each prereg. *(Superseded in v1.9.2 by F13 + F9 stack R@5 = 86.8 on oracle.)*
+
+### What's new in v1.9.0
+
+- **F6 reranker hardening shipped.** New `RerankerFn` seam in `hybridSearch` with three reranker tracks: Track 1 features (`MemoryEntry`-level signals, no external deps), Track 2 cross-encoder (MS-MARCO MiniLM via optional `@xenova/transformers`, identity-fallback on load failure), Track 3 LLM (env-gated skeleton against an OpenAI-compatible endpoint). Opt in via `hippo recall --reranker <name>`.
+- **Workload-validity verdicts on the LongMemEval sweep** (`docs/evals/2026-05-10-f6-reranker-result.md`, prereg `docs/evals/2026-05-10-f6-reranker-prereg.md`): Gate-A (firing rate, binding) PASS for the features track, PASS-with-caveat for cross-encoder (500/500 invocations all took the identity-fallback branch — HF model download was blocked in the test environment, so this is NOT a real cross-encoder evaluation). Gate-B (hyperparameter discrimination, binding) FAIL — features_topk{20,50,100} produced byte-identical R@K, so no per-hyperparameter R@5 effect is claimed.
+- **Roadmap R@5 ≥ 85% target NOT met on the workload tested.** Observed R@5 = 75.4% (features, all three top-K settings) and 75.6% (baseline). Per the prereg this is descriptive characterisation, not a binding gate; the mechanism ships, and a real attempt at the target requires either a real cross-encoder evaluation (HF access) or a richer ingest path that populates entry-level reranker signals.
+- **This release does not re-assert the retracted −10pp magnitude.** Per `docs/RETRACTION.md`. The dlPFC goal-stack cumulative-null status (`docs/RETRACTION.md:94-113`) is independent of this release.
+
 ### What's new in v1.8.1
 
 - **v1.8 prereg's v1.9 LongMemEval cross-validation pre-commitment RETRACTED.** Outside-voice review on two iterations of the v1.9 plan found six structural barriers (canonical harness bypasses the boost path; ingest tag namespace excludes content-derived stems; pushGoal API field mismatch; depth-cap suspension; trigger AND clause unreachable; workload-validity gate ceremonial). Per Root Cause Over Patches, public retraction over re-architecture. **This release does not re-assert the retracted −10pp magnitude.** Per `docs/RETRACTION.md`.

package/bin/hippo.js

@@ -1,2 +1,2 @@
-#!/usr/bin/env node
-import '../dist/cli.js';
+#!/usr/bin/env node
+import '../dist/cli.js';

package/dist/cli.js

@@ -64,6 +64,7 @@ import { runFeatureEval, formatResult, resultToBaseline, detectRegressions } fro
 import { refineStore } from './refine-llm.js';
 import { wmPush, wmRead, wmClear, wmFlush } from './working-memory.js';
 import { multihopSearch } from './multihop.js';
+import { getReranker } from './rerankers/index.js';
 import { computeSalience } from './salience.js';
 import { computeAmbientState, renderAmbientSummary } from './ambient.js';
 import { listDlq, replayDlqEntry } from './connectors/slack/dlq.js';
@@ -701,17 +702,21 @@ async function cmdRecall(hippoRoot, query, flags) {
             physicsConfig: config.physics,
             minResults,
             scope: recallActiveScope,
+            includeSuperseded,
+            asOf,
         });
     }
     else if (hasGlobal) {
         // Use searchBothHybrid for merged results with embedding support
         results = await searchBothHybrid(query, hippoRoot, globalRoot, {
             budget, mmr: mmrEnabled, mmrLambda, localBump, minResults, scope: recallActiveScope, tenantId,
+            includeSuperseded, asOf,
         });
     }
     else {
         results = await hybridSearch(query, localEntries, {
             budget, hippoRoot, mmr: mmrEnabled, mmrLambda, minResults, scope: recallActiveScope,
+            includeSuperseded, asOf,
         });
     }
     // ACC EVC-adaptive recall (RESEARCH.md §PFC.ACC). When the initial top-K is
@@ -818,6 +823,39 @@ async function cmdRecall(hippoRoot, query, flags) {
         })
             .sort((a, b) => b.score - a.score);
     }
+    // F6 reranker pass (docs/plans/2026-05-10-f6-reranker-hardening.md). When
+    // --reranker <name> is set, look up the reranker fn from the registry
+    // (src/rerankers/index.ts) and apply it to the top-K candidates. The
+    // reranker reorders (and may rescale) results; the post-budget set is
+    // returned. Default off; opt-in via --reranker <cross-encoder|llm>. The
+    // structurally similar --rerank-utility block above is the OFC MVP and is
+    // independent — both can run in the same recall, with --rerank-utility
+    // applied first. Available rerankers: cross-encoder, llm (see
+    // src/rerankers/index.ts). The Track 1 `features` reranker was removed in
+    // v1.9.1 per the F10 HARD RETRACTION; it is no longer a valid value.
+    const rerankerName = flags['reranker'] !== undefined ? String(flags['reranker']).trim() : '';
+    if (rerankerName) {
+        const rerankerFn = getReranker(rerankerName);
+        if (rerankerFn) {
+            const topK = flags['reranker-top-k'] !== undefined
+                ? parseInt(String(flags['reranker-top-k']), 10)
+                : 50;
+            const head = results.slice(0, topK);
+            const tail = results.slice(topK);
+            const rerankInput = head.map((r, i) => ({ ...r, preRerankRank: i + 1 }));
+            const reranked = await rerankerFn(query, rerankInput, { topK });
+            // Copy rerankScore into score so downstream blocks (--goal, goal-stack,
+            // salience) that sort by `r.score` honor the reranker's order rather
+            // than unwinding it. Original score is preserved on rerankScore's
+            // input, but downstream sorters key on `score`.
+            const withPostRank = reranked.map((r, i) => ({
+                ...r,
+                score: r.rerankScore,
+                postRerankRank: i + 1,
+            }));
+            results = [...withPostRank, ...tail];
+        }
+    }
     // dlPFC goal-conditioned recall MVP (RESEARCH.md §PFC.dlPFC). When --goal
     // <tag> is set, memories whose `tags` array contains the goal tag receive
     // a 1.5x score boost and results are re-sorted. The full dlPFC spec
@@ -4649,6 +4687,12 @@ Commands:
                            = score * (0.5 + 0.5 * strength) * (1 - cost_factor)
                            where cost_factor = min(0.3, tokens / 10000). Re-sorts
                            results by utility. Default off. RESEARCH.md §PFC.OFC.
+    --reranker <name>      Apply a reranker pass after retrieval
+                           (cross-encoder|llm). Looks up the named
+                           reranker from src/rerankers/index.ts and re-orders
+                           the top-K candidates. Default unset (no reranker).
+                           See docs/plans/2026-05-10-f6-reranker-hardening.md.
+    --reranker-top-k <n>   Cap candidates passed to the reranker (default 50).
     --goal <tag>           dlPFC goal-conditioned recall: memories tagged with
                            the goal tag get a 1.5x score boost and results are
                            re-sorted. Default off. RESEARCH.md §PFC.dlPFC.