@functional-systems/srcx 0.1.0-dev.13 → 0.1.0-dev.15

package/CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## Unreleased
 
+- Cross-check the retained CosQA lexical baseline with an independent offline
+  BM25 ranker and the same StandardAnalyzer tokenization.
+
+- Add pinned CoIR CoSQA and MTEB CodeSearchNet Python/Go retrieval baselines,
+  sharing srcx query construction and using trec_eval metrics on official labels.
+
 - Add a bounded local Qwen reranking diagnostic with frozen inputs, durable
   per-pair reservations and offline source-selection reporting.
 
@@ -53,6 +59,9 @@
 
 ### Fixed
 
+- Count failed and unsupported public-benchmark queries separately, retaining
+  both in the ranking metric denominators.
+
 - Fetch omitted managed vectors from query responses through the same immutable
   Tag, preserving hit order/scores and requiring exact non-vector payload equality.
 

package/README.md

@@ -165,6 +165,10 @@ independently reviewed benchmark.
 The [local reranking diagnostic](https://github.com/lambdadb/srcx/blob/develop/eval/RERANK-RESULTS.md)
 compares Qwen selection on the same saved candidates, with separate offline
 coverage and compute accounting.
+The [public retrieval baseline](https://github.com/lambdadb/srcx/blob/develop/eval/PUBLIC-BENCHMARK.md)
+uses pinned CoIR CoSQA and MTEB CodeSearchNet Python/Go tasks with official
+labels and trec_eval metrics. It evaluates retrieval separately from Git import,
+chunking and CLI reads.
 New evaluation roots retain [safe command failure diagnostics](https://github.com/lambdadb/srcx/blob/develop/eval/COMMAND-DIAGNOSTICS.md)
 for explicit recovery.
 

package/VALIDATION.md

@@ -2,6 +2,27 @@
 
 Date: 2026-09-25 (Asia/Seoul).
 
+## Independent CosQA BM25 cross-check
+
+The [independent cross-check](eval/COSQA-BM25-RESULTS.md) reproduces lexical
+quality after aligning score ties and documents duplicate-ID sensitivity in the
+official task. Its supplemental content-equivalence metrics are distinct from
+the unchanged official-label baseline.
+
+## Public code retrieval baseline
+
+At frozen executable `03ac468`, the pinned CoIR CosQA and MTEB CodeSearchNet
+Python/Go baseline validated all 22,604 documents and managed vectors in immutable
+Tags and collected 7,473 successful searches. Nine Python inputs exceeded the
+existing query limit; all 27 mode/query outcomes remained zero-scored in the
+2,500-query denominator for each mode. There were no API failures or retries.
+
+Semantic nDCG@10 exceeded lexical and the current RRF hybrid on all three tasks.
+This is an external-task retrieval baseline, not a full CoIR average, Git/CLI
+integration validation or an agent coding-success benchmark. Product defaults
+are unchanged. See [full metrics, usage and limitations](eval/PUBLIC-BENCHMARK-RESULTS.md)
+and the [fixed protocol](eval/PUBLIC-BENCHMARK.md).
+
 ## Evidence boundary
 
 The initial implementation passed local fixtures/fault injection/SDK transport checks,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@functional-systems/srcx",
-  "version": "0.1.0-dev.13",
+  "version": "0.1.0-dev.15",
   "private": false,
   "description": "Version-aware code search on LambdaDB",
   "type": "module",
@@ -62,5 +62,5 @@
     "url": "https://github.com/lambdadb/srcx/issues"
   },
   "license": "Apache-2.0",
-  "gitHead": "a94577f19b93ecbeed390338bfae699efe99faa5"
+  "gitHead": "3b8453018d115863711b29e141565ea104152b1e"
 }