@functional-systems/srcx 0.1.0-dev.6 → 0.1.0-dev.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/CHANGELOG.md +4 -2
  2. package/README.md +5 -0
  3. package/VALIDATION.md +27 -0
  4. package/package.json +5 -3
package/CHANGELOG.md CHANGED
@@ -2,11 +2,11 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ - Default CLI search/read evaluation across two pinned public repositories, with
6
+ alternative answer sets, exact source verification, and complete stdout accounting.
5
7
  - Internal reproducible lexical chunking evaluation with pinned source labels,
6
8
  equal path-only enrichment, isolated live indexes, and token-budget metrics.
7
9
 
8
- ## Unreleased
9
-
10
10
  ### Added
11
11
 
12
12
  - Persistent one-to-one Git branch / Collection Branch mapping, branch-local
@@ -27,6 +27,8 @@
27
27
 
28
28
  ### Fixed
29
29
 
30
+ - Reject an unprepared CLI evaluation run before creating its root/lock so a
31
+ subsequent offline preparation can use the same path.
30
32
  - Pin branch control before enumerating versions so a concurrent publication
31
33
  cannot advance the selector beyond the version list being read.
32
34
  - Reject live checkpoint reuse across different code/harness inputs and preserve
package/README.md CHANGED
@@ -84,6 +84,11 @@ window chunks on a fixed public corpus and 16 pinned investigation queries.
84
84
  `npm run eval:prepare` builds and validates the comparison offline. Live evaluation
85
85
  is an explicit separate operation and retains isolated evaluation Collections.
86
86
 
87
+ The [default CLI workflow evaluation](https://github.com/lambdadb/srcx/blob/develop/eval/CLI-WORKFLOW.md)
88
+ adds a second public repository and verifies real `search` → `read` results with
89
+ the unchanged CLI preset. It uses normal repository Collections and separate local
90
+ state; see its documented effects before opting into the live run.
91
+
87
92
  ## Explicit live acceptance run
88
93
 
89
94
  The live harness accepts `LAMBDADB_BASE_URL`, `LAMBDADB_PROJECT_NAME`, and
package/VALIDATION.md CHANGED
@@ -12,6 +12,33 @@ concurrency, or retrieval-quality benchmark. npm publication evidence is recorde
12
12
  separately below. The persistent Git branch changes have a separate synthetic
13
13
  live run recorded next; the earlier live runs do not validate this new path.
14
14
 
15
+ ## Default CLI search/read evaluation
16
+
17
+ The follow-up uses actual built CLI subprocesses with the unchanged default
18
+ syntax/path-scope-symbol preset across pinned public `srcx` and `lambdadb-cli`
19
+ commits. At harness commit `57b63e1cdbf7ec722b95bb0f40ac0ef5039af3e8`, both
20
+ publications validated and all **136 search results/handles** and **76 read
21
+ responses** matched their pinned source, version and citation. No embeddings
22
+ were used. The run completed at `2026-09-24T20:01:58.865Z`.
23
+
24
+ The frozen 16-question suite recovered complete labeled evidence for **14/16**.
25
+ The two remaining cases reflect a one-byte blank-line boundary and an omitted
26
+ alternative document/code answer, not demonstrated source-integrity failures.
27
+ Mean full CLI stdout was **7,383 tokens** per query. This fixed-five-read protocol
28
+ is distinct from the earlier full-chunk/token-budget comparison; neither measures
29
+ general task success. Labels remain assistant-authored without independent human
30
+ review. See [eval/CLI-RESULTS.md](https://github.com/lambdadb/srcx/blob/develop/eval/CLI-RESULTS.md)
31
+ for per-repository scores, inspected misses and evidence boundaries, and
32
+ [eval/CLI-WORKFLOW.md](https://github.com/lambdadb/srcx/blob/develop/eval/CLI-WORKFLOW.md)
33
+ for reproduction and normal Collection effects.
34
+
35
+ Local Node 24.15.0 passed **54 tests**, typechecking, formatting/version checks and
36
+ the installed-package CLI contract. Node 22/24 CI passed at the harness commit.
37
+ Live execution used built CLI subprocesses; package validation used a loopback
38
+ fixture. A completed rerun preserved the original report bytes and timestamp.
39
+ Raw evidence and recovery journals remain under `.srcx/cli-eval/` in the validation
40
+ worktree; the older pilot's original report remains unchanged.
41
+
15
42
  ## Retrieval evaluation harness
16
43
 
17
44
  The internal pilot adds five checks for unique source-range scoring, token-budget
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@functional-systems/srcx",
3
- "version": "0.1.0-dev.6",
3
+ "version": "0.1.0-dev.7",
4
4
  "private": false,
5
5
  "description": "Version-aware code search on LambdaDB",
6
6
  "type": "module",
@@ -35,7 +35,9 @@
35
35
  "test:package": "node scripts/test-package.mjs",
36
36
  "test:live:branches": "npm run build && node --env-file=.env.local scripts/live-branches.mjs",
37
37
  "eval:prepare": "npm run build && node scripts/retrieval-eval.mjs prepare",
38
- "eval:run": "node scripts/retrieval-eval.mjs run"
38
+ "eval:run": "node scripts/retrieval-eval.mjs run",
39
+ "eval:cli:prepare": "npm run build && node scripts/cli-eval.mjs prepare",
40
+ "eval:cli:run": "node scripts/cli-eval.mjs run"
39
41
  },
40
42
  "dependencies": {
41
43
  "@functional-systems/lambdadb": "0.5.1",
@@ -58,5 +60,5 @@
58
60
  "url": "https://github.com/lambdadb/srcx/issues"
59
61
  },
60
62
  "license": "Apache-2.0",
61
- "gitHead": "8c0d1656d4d66a6461a32e9072b2e5f7fda9cec1"
63
+ "gitHead": "c619d643dfd2d945fff1bc470cbfc5ba339dda77"
62
64
  }