sigmap 8.26.1 → 8.26.2

package/CHANGELOG.md

@@ -10,6 +10,18 @@ Format: [Semantic Versioning](https://semver.org/)
 
 ---
 
+## [8.26.2] — 2026-08-18
+
+Patch release — **"Honest Harness" (#522)**: the benchmark suite's cross-suite instability is diagnosed, fixed, and gated — and the headline retrieval number honestly resettles on the now-stable harness.
+
+### Fixed
+- **Benchmark cross-suite determinism (#522, PR #524):** two measured root causes. (1) The quality suite carried a stale 13-entry local copy of the retrieval harness's 21-entry `CONFIG_OVERRIDES` table and regenerated every repo unconditionally — the missing entries (express, flask, spring-petclinic, serilog) got default-srcDirs contexts, silently overwriting the canonical ones (measured: honest hit@5 77.6% → 66.4% after a quality run; flask 0.875 → 0). (2) The 20-task `retrieval` set scores against the **live SigMap repo**, whose context legitimately changes every release (measured: 0.85 → 0.55 over three releases) — real development drift, previously indistinguishable from harness noise. Fixes: a single canonical `benchmarks/config-overrides.json` loaded by both suites; the quality suite now mirrors the retrieval harness's apply/restore semantics exactly (always apply · regenerate · restore in `finally`); the honest report labels the self-repo row (`selfRepo: true`, `*` + note in output). New gate `npm run validate:benchmark-determinism` runs honest → quality → honest and fails on any divergence — verified green post-fix (identical at 77.6% / 125 tasks). 5 CI-safe source-level guard tests.
+
+### Changed
+- **Headline honesty, again:** with the harness stable, the honest baseline resettles — the previously published 82.4% hit@5 included ~5pt of unlabeled self-repo drift. The refreshed numbers in this release are the first produced under the determinism gate. 134 test files.
+
+---
+
 ## [8.26.1] — 2026-08-18
 
 Patch release — **"Trust Quick Wins II" (G1)**: the extraction layer gets the same honesty treatment the benchmarks got in v8.19.

package/README.md

@@ -59,7 +59,7 @@ That map is exactly what agentic grep is worst at: reproducible, auditable conte
 
 **Proof it pays off** (full benchmark below):
 <!--SM:whyMetrics-->
-- **82.2% hit@5** — right file in top 5 results (vs 44.0% single-shot grep baseline — 1.87× lift)
+- **82.2% hit@5** — right file in top 5 results (vs 44.0% single-shot grep baseline — 1.76× lift)
 - **96.8% token reduction** — average across 21 real repos
 - **64.8% task-success proxy** — modeled from retrieval tiers, not measured LLM sessions
 - **1.53 prompts per task** — down from 2.84 (46.1% fewer retries, modeled)
@@ -123,9 +123,9 @@ Ask → Rank → Context → Validate → Judge → Learn
 <!--SM:benchmarkBlock-->
 ```
 Benchmark : sigmap-v8.26-main (21 repositories, including R language)
-Date      : 2026-08-17
+Date      : 2026-08-18
 
-Hit@5          : 82.2%   (grep-agent baseline 44.0%  — 1.87× lift)
+Hit@5          : 82.2%   (grep-agent baseline 44.0%  — 1.76× lift)
 Token reduction: 96.8%   (across 21 repos)
 Prompt reduction : 46.1% (2.84 → 1.53 prompts per task, modeled)
 Task success   : 64.8%   (proxy — modeled from retrieval tiers)

package/gen-context.js

@@ -15195,7 +15195,7 @@ __factories["./src/mcp/server"] = function(module, exports) {
 
   const SERVER_INFO = {
     name: 'sigmap',
-    version: '8.26.1',
+    version: '8.26.2',
     description: 'SigMap MCP server — code signatures on demand',
   };
 
@@ -20807,7 +20807,7 @@ function __tryGit(args, opts = {}) {
   catch (_) { return ''; }
 }
 
-const VERSION = '8.26.1';
+const VERSION = '8.26.2';
 const MARKER = '\n\n## Auto-generated signatures\n<!-- Updated by gen-context.js -->\n';
 
 function requireSourceOrBundled(key) {

package/llms-full.txt

@@ -11,17 +11,17 @@ ranking keeps the relevant context in scope (cutting tokens ~97% as a side
 effect), with no LLM calls, embeddings, or vector database. Works with Claude,
 Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
 
-# Version: 8.26.1 | Benchmark: sigmap-v8.26-main (2026-08-17)
+# Version: 8.26.2 | Benchmark: sigmap-v8.26-main (2026-08-18)
 # Source: auto-generated from package.json, version.json, benchmarks/latest.json, src/mcp/tools.js, src/config/defaults.js
 # Regenerate: npm run generate:llms   |   Validate: npm run validate:llms
 
 ---
 
-## Core metrics (benchmark: sigmap-v8.26-main, 2026-08-17)
+## Core metrics (benchmark: sigmap-v8.26-main, 2026-08-18)
 
 | Metric | Without SigMap | With SigMap |
 |--------|----------------|-------------|
-| Retrieval hit@5 | 44.0% (single-shot grep) | 82.2% (1.87× lift) |
+| Retrieval hit@5 | 44.0% (single-shot grep) | 82.2% (1.76× lift) |
 | Token reduction | — | 96.8% average |
 | Task-success proxy (modeled) | — | 64.8% |
 | Prompts per task | 2.84 | 1.53 (46.1% fewer) |

package/llms.txt

@@ -11,7 +11,7 @@ ranking keeps the relevant context in scope (cutting tokens ~97% as a side
 effect), with no LLM calls, embeddings, or vector database. Works with Claude,
 Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
 
-# Version: 8.26.1 | Benchmark: sigmap-v8.26-main (2026-08-17)
+# Version: 8.26.2 | Benchmark: sigmap-v8.26-main (2026-08-18)
 # Source: auto-generated from package.json, version.json, benchmarks/latest.json, src/mcp/tools.js, src/config/defaults.js
 # Regenerate: npm run generate:llms   |   Validate: npm run validate:llms
 
@@ -23,9 +23,9 @@ Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
 - No blast-radius awareness before editing a hub file — `--impact` shows every file a change touches.
 - Pasted stack traces, CI logs, and JSON bloat the prompt — `squeeze` minimizes them and enriches the top frame from the symbol index.
 
-## Core metrics (benchmark: sigmap-v8.26-main, 2026-08-17)
+## Core metrics (benchmark: sigmap-v8.26-main, 2026-08-18)
 
-- hit@5 retrieval: 82.2% vs 44.0% single-shot grep baseline (1.87× lift)
+- hit@5 retrieval: 82.2% vs 44.0% single-shot grep baseline (1.76× lift)
 - Token reduction: 96.8% average across benchmark repos
 - Task-success proxy: 64.8% (modeled from retrieval tiers, not measured LLM sessions)
 - Prompts per task: 1.53 vs 2.84 baseline (46.1% fewer, modeled)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sigmap",
-  "version": "8.26.1",
+  "version": "8.26.2",
   "description": "The deterministic, verifiable grounding layer for AI code work — a zero-dependency signature-and-evidence map that grounds Claude, Cursor, Copilot, Aider, Windsurf, local LLMs & MCP agents against your real code (repo + installed libraries) so they stop hallucinating files, imports & APIs. Runs offline via npx; byte-stable output; ~97% token reduction as proof.",
   "main": "packages/core/index.js",
   "exports": {
@@ -34,6 +34,7 @@
     "benchmark:centrality-blend": "node scripts/run-centrality-blend-benchmark.mjs --save",
     "benchmark:surface-enrichment": "node scripts/run-surface-enrichment-benchmark.mjs --save",
     "validate:squeeze": "node scripts/run-squeeze-benchmark.mjs --gate",
+    "validate:benchmark-determinism": "node scripts/check-benchmark-determinism.mjs --cross-suite",
     "health": "node gen-context.js --health",
     "map": "node gen-project-map.js",
     "mcp": "node gen-context.js --mcp",

package/packages/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sigmap-cli",
-  "version": "8.26.1",
+  "version": "8.26.2",
   "description": "SigMap CLI wrapper — thin adapter for programmatic CLI invocation",
   "main": "index.js",
   "keywords": [

package/packages/core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sigmap-core",
-  "version": "8.26.1",
+  "version": "8.26.2",
   "description": "SigMap core library — zero-dependency code signature extraction, retrieval, and security scanning",
   "main": "index.js",
   "keywords": [

package/src/mcp/server.js

@@ -18,7 +18,7 @@ const { readContext, searchSignatures, getMap, createCheckpoint, getRouting, exp
 
 const SERVER_INFO = {
   name: 'sigmap',
-  version: '8.26.1',
+  version: '8.26.2',
   description: 'SigMap MCP server — code signatures on demand',
 };