@functional-systems/srcx 0.1.0-dev.12 → 0.1.0-dev.14

package/CHANGELOG.md

@@ -2,6 +2,15 @@
 
 ## Unreleased
 
+- Add pinned CoIR CoSQA and MTEB CodeSearchNet Python/Go retrieval baselines,
+  sharing srcx query construction and using trec_eval metrics on official labels.
+
+- Add a bounded local Qwen reranking diagnostic with frozen inputs, durable
+  per-pair reservations and offline source-selection reporting.
+
+- Prepare offline reranking inputs from verified retained Click/Cobra candidates,
+  with separate evaluator labels, original ranks and source provenance.
+
 - Add a pending human review worksheet and fixed-candidate reranking experiment plan.
 
 - Run bounded, explicitly unreviewed Click/Cobra retrieval diagnostics with managed small and unchanged source labels.
@@ -47,6 +56,9 @@
 
 ### Fixed
 
+- Count failed and unsupported public-benchmark queries separately, retaining
+  both in the ranking metric denominators.
+
 - Fetch omitted managed vectors from query responses through the same immutable
   Tag, preserving hit order/scores and requiring exact non-vector payload equality.
 

package/README.md

@@ -162,6 +162,13 @@ show mixed gains and regressions; small remains the initial managed choice.
 unreviewed Python/Go questions. The [completed development diagnostic](https://github.com/lambdadb/srcx/blob/develop/eval/TRANSFER-RESULTS.md)
 compares all three modes with managed small and text fallback; it is not an
 independently reviewed benchmark.
+The [local reranking diagnostic](https://github.com/lambdadb/srcx/blob/develop/eval/RERANK-RESULTS.md)
+compares Qwen selection on the same saved candidates, with separate offline
+coverage and compute accounting.
+The [public retrieval baseline](https://github.com/lambdadb/srcx/blob/develop/eval/PUBLIC-BENCHMARK.md)
+uses pinned CoIR CoSQA and MTEB CodeSearchNet Python/Go tasks with official
+labels and trec_eval metrics. It evaluates retrieval separately from Git import,
+chunking and CLI reads.
 New evaluation roots retain [safe command failure diagnostics](https://github.com/lambdadb/srcx/blob/develop/eval/COMMAND-DIAGNOSTICS.md)
 for explicit recovery.
 

package/VALIDATION.md

@@ -2,6 +2,20 @@
 
 Date: 2026-09-25 (Asia/Seoul).
 
+## Public code retrieval baseline
+
+At frozen executable `03ac468`, the pinned CoIR CosQA and MTEB CodeSearchNet
+Python/Go baseline validated all 22,604 documents and managed vectors in immutable
+Tags and collected 7,473 successful searches. Nine Python inputs exceeded the
+existing query limit; all 27 mode/query outcomes remained zero-scored in the
+2,500-query denominator for each mode. There were no API failures or retries.
+
+Semantic nDCG@10 exceeded lexical and the current RRF hybrid on all three tasks.
+This is an external-task retrieval baseline, not a full CoIR average, Git/CLI
+integration validation or an agent coding-success benchmark. Product defaults
+are unchanged. See [full metrics, usage and limitations](eval/PUBLIC-BENCHMARK-RESULTS.md)
+and the [fixed protocol](eval/PUBLIC-BENCHMARK.md).
+
 ## Evidence boundary
 
 The initial implementation passed local fixtures/fault injection/SDK transport checks,
@@ -12,6 +26,22 @@ concurrency, or retrieval-quality benchmark. npm publication evidence is recorde
 separately below. The persistent Git branch changes have a separate synthetic
 live run recorded next; the earlier live runs do not validate this new path.
 
+## Local reranking diagnostic
+
+At frozen executable `ba9fa0f8ad5c57cb54ec0483d05faead78acf2c1`, the local
+Qwen3-Reranker-0.6B treatment scored all 455 retained pairs without failures or
+retries. Offline complete source evidence among five selections changed from
+lexical **7/16 to 8/16**, semantic **5/16 to 9/16**, hybrid **8/16 to 10/16**.
+There were no complete-answer losses; partial evidence declined in two pools.
+No new CLI reads were made and the original labels remain unreviewed.
+
+The run used 391,713 Qwen input tokens and 84.59 seconds on local MPS, with zero
+paid API usage. Completed replay preserved score bytes/timestamps without model
+loading. Local validation passed 86 Node tests, three Python protocol checks and
+three installed-package checks; frozen Node 22/24 CI passed. See [results and
+limitations](eval/RERANK-RESULTS.md), [frozen protocol](eval/RERANK-RUN.md) and
+[preflight](eval/rerank-qwen-preflight.json) for provenance and bounds.
+
 ## Click/Cobra development diagnostic
 
 At frozen harness `4f7c865055749c82641015a2302983b034877e23`, 48 actual CLI searches

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@functional-systems/srcx",
-  "version": "0.1.0-dev.12",
+  "version": "0.1.0-dev.14",
   "private": false,
   "description": "Version-aware code search on LambdaDB",
   "type": "module",
@@ -62,5 +62,5 @@
     "url": "https://github.com/lambdadb/srcx/issues"
   },
   "license": "Apache-2.0",
-  "gitHead": "48eb996a9be7b9d56582a5bb9fda4637449b7db8"
+  "gitHead": "5a214d6124a84f831f4595c159576d319b94f7fc"
 }