@functional-systems/srcx 0.1.0-dev.17 → 0.1.0-dev.19

package/CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## Unreleased
 
+- Record three developer search/read investigations with concrete change maps,
+  bounded live usage and source verification, separate from ranking benchmarks.
+
+- Refresh implemented/remaining scope and verify the published dev.17 package
+  through a fresh npm consumer and bounded live first-use workflow.
+
 - Add opt-in local Qwen search reranking with bounded candidate pools, pinned
   source evidence, separate scores and per-command timing.
 

package/DESIGN.md

@@ -1,10 +1,11 @@
 # srcx: Initial Implementation Design
 
-Status: initial CLI implemented with local tests and live synthetic A/B validation
-on LambdaDB. See [README.md](README.md) for runnable interfaces and
+Status: CLI implemented with persistent Git branch tracking, managed semantic/hybrid
+search and optional local Qwen reranking. Local, live and public-benchmark evidence
+have separate scopes. See [README.md](README.md) for runnable interfaces and
 [VALIDATION.md](VALIDATION.md) for the bounded live evidence and remaining scope.
 
-Updated: 2026-09-25.
+Updated: 2026-09-26.
 
 ## 1. Purpose, decisions, and implementation defaults
 
@@ -557,10 +558,10 @@ baseline, not a proven code-specific ranking solution. An initial enrichment
 candidate preserves original identifiers while adding camelCase/snake_case terms;
 pin and evaluate that transformation before adopting it as the preset.
 
-The opt-in managed preset adds indexed text `embeddingText` and a managed
-`embedding` vector field with provider `openai`, model `text-embedding-3-small`,
-1536 dimensions and cosine similarity. The source field, dimensions and similarity
-are nested under the vector field's `embedding` configuration; there are no
+The opt-in managed presets add indexed text `embeddingText` and a managed
+`embedding` vector field with provider `openai`: `text-embedding-3-small` uses
+1536 dimensions and `text-embedding-3-large` uses 3072, both with cosine similarity.
+The source field, dimensions and similarity are nested under the vector field's `embedding` configuration; there are no
 top-level dimension/similarity properties. Collection discovery verifies the
 complete resolved schema and pinned preset. Missing `embeddingText` on skipped
 chunks, files and manifests means no vector is generated for those records. The `none` preset omits the vector index entirely.
@@ -769,12 +770,22 @@ shape and payload integrity, not deterministic numerical reproduction by OpenAI.
 Search remains lexical by default. Explicit `--mode semantic` uses managed
 `knn.queryText`; `--mode hybrid` combines the existing lexical query and that kNN
 query using LambdaDB RRF. Both legs enforce `kind=chunk` and the same optional
-path/language filters; kNN uses a prefilter and `k=limit`. The lexical leg keeps
+path/language filters; kNN uses a prefilter and `k=limit`, or `k=candidates`
+when optional reranking requests a wider pool. The lexical leg keeps
 nonembedded chunks eligible. Query/fetch/list request vectors explicitly so the
 same validation applies to pinned result handles. When a deployment omits managed
 vectors from list responses, fetch the affected IDs from that same immutable Tag
 and require identical non-vector payloads before vector validation. Source reads still use the
-original file bytes, never enriched text. No relevance gain is claimed yet.
+original file bytes, never enriched text. Bounded retrieval and reranking results
+are recorded in [VALIDATION.md](VALIDATION.md); they do not establish a universal
+winner or authorize default changes.
+
+Optional `--rerank qwen` scores complete verified chunk bytes with a pinned local
+Qwen3-Reranker-0.6B revision, preserves candidate identity and retrieval order on
+ties, and returns a separate reranker score. Setup is explicit, inference is
+offline, and failures do not silently fall back. Every CLI invocation loads its
+own model. See [README.md](README.md#optional-local-qwen-reranker) for limits and
+runtime setup; this path does not change stored Collections or embedding presets.
 
 Managed source text and semantic/hybrid query text are sent through LambdaDB to
 OpenAI and incur embedding usage plus normal LambdaDB operations/storage. Dry-run
@@ -1067,15 +1078,18 @@ latency, and operational cost are separate measurements.
 
 ## 12. Remaining decisions and inspected evidence
 
+The initial parser/tokenizer presets, managed embedding choices and CLI retrieval
+modes are implemented and have bounded evaluation records. The following are
+future decisions or per-run requirements, not blockers awaiting initial design:
+
 | Decision | When needed |
 | --- | --- |
-| First real repositories and parser priority | Real-repository evaluation; defaults do not block fixtures |
-| Exact parser/tokenizer/preset versions, enrichment and embedding skip rules | Bootstrap and retrieval evaluation; pin before publishing a corpus |
-| Chunk size/overlap and syntax-versus-window tradeoff | Section 8 pilot after retrieval works; current values are starting points |
-| Embedding model/dimensions/provider, data destination, spending cap | Before any real source upload or paid experiment |
-| LambdaDB endpoint/project and credentials | Provisioning and live fixture run |
-| Authoritative Git-tag source and observed-ref freshness policy | Manual Alias synchronization |
-| Concurrent multi-host writers, retention/cleanup, configuration migration | Later internal operation, outside first CLI |
+| Additional parser languages and production repositories | Onboarding source beyond the currently tested coverage |
+| Quality/latency thresholds and any default changes | After actual development-task use; before stable rollout |
+| New embedding/reranking providers or chunking policies | A concrete unmet retrieval need; Jev access remains deferred |
+| Source authorization, endpoint/project, credentials and spending limits | Before every new corpus upload or paid experiment |
+| Authoritative Git-tag source and observation freshness | Automatic synchronization or pruning |
+| Multi-host writers, retention/cleanup and configuration migration | Later internal operation, outside the first CLI |
 
 Sources and evidence used during this design discussion:
 
@@ -1112,13 +1126,13 @@ Sources and evidence used during this design discussion:
 - [Tree-sitter parsing](https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html):
   source ranges and encodings; verify the selected binding during implementation.
 
-Development checkout references identify the inspected files, not runtime
-dependencies. No
-application tests, source uploads, embeddings, or live LambdaDB validation have
-been performed as part of writing this document.
+Development checkout references identify the files inspected during the original
+design discussion, not runtime dependencies. That design-only review did not run
+application tests, source uploads, embeddings or live LambdaDB checks. Subsequent
+implementation and validation evidence is recorded in [VALIDATION.md](VALIDATION.md).
 
-Document validation: the revised lexical `indexConfigs` JSON, description, four
-reserved metadata labels, and one optional context label passed the local built
+Historical design validation (September 25): the revised lexical `indexConfigs`
+JSON, description, four reserved metadata labels, and one optional context label passed the local built
 SDK's Collection-create input validator on September 25. All three JSON examples,
 the illustrative source span, then-local links, code fences, and whitespace also passed
 document checks. These checks do not establish server acceptance, marker ordering,

package/README.md

@@ -183,6 +183,11 @@ or fallback. Empty candidate pools skip model execution. Existing search default
 and managed embedding usage rules remain unchanged; wider pools increase source
 verification work. See the [integration validation](VALIDATION.md#local-reranker-integration).
 
+For practical search → read → change-planning examples, see the
+[developer workflow pilot](https://github.com/lambdadb/srcx/blob/develop/eval/DEVELOPER-WORKFLOW-PILOT.md).
+It follows three maintenance investigations through related source and tests,
+including the limitations of previews and optional reranking.
+
 ## Evaluate retrieval quality
 
 The internal [lexical chunking pilot](https://github.com/lambdadb/srcx/blob/develop/eval/README.md) compares syntax-aware and
@@ -265,40 +270,46 @@ run and any pending journal. Its per-import deadline defaults to 300 seconds and
 
 ## Connected workflow
 
+Use the installed `srcx` binary below. For a source checkout, build first and
+replace `srcx` with `node dist/cli.js`.
+
 These commands create and write LambdaDB resources. Select an approved project
-and source scope before running them. Start live validation with the synthetic
-fixture, not a production repository.
+and source scope before running them. Start live validation with a small synthetic
+fixture. `repo add` returns a `collection` value: use that value as `<name>` below
+to disambiguate repositories with multiple presets. If isolating a first-use run,
+set `SRCX_CONFIG` and `SRCX_STATE_DIR` to fresh paths before `configure`; keep
+credentials in the shell and preserve these paths for later result reads.
 
 ```sh
 # Supply LAMBDADB_API_KEY through your shell or secret manager.
-node dist/cli.js configure --endpoint https://api.lambdadb.ai --project <project>
-node dist/cli.js doctor
-node dist/cli.js repo add --path /path/to/repo \
+srcx configure --endpoint https://api.lambdadb.ai --project <project>
+srcx doctor
+srcx repo add --path /path/to/repo \
   --description "Repository source, tests, and documentation for code search." \
   --tag team=search
-node dist/cli.js repo list
-node dist/cli.js repo show --repo <name>
+srcx repo list
+srcx repo show --repo <name>
 
 # Import the exact artifact reviewed in the credential-free preview.
-node dist/cli.js import --repo <name> --artifact /tmp/srcx-build-a
-node dist/cli.js versions --repo <name>
+srcx import --repo <name> --artifact /tmp/srcx-build-a
+srcx versions --repo <name>
 
 # Keep a Git branch connected to its Collection Branch.
-node dist/cli.js import --repo <name> --ref develop
-node dist/cli.js search --repo <name> --version develop --query retry
-node dist/cli.js resolve --repo <name> --ref refs/heads/develop
-node dist/cli.js search --repo <name> --version <full-commit-A> --query retry
-node dist/cli.js read --result <result-id> --context 20
-node dist/cli.js read --result <result-id> --full-file
-node dist/cli.js read --repo <name> --version <full-commit-A> \
+srcx import --repo <name> --ref develop
+srcx search --repo <name> --version develop --query retry
+srcx resolve --repo <name> --ref refs/heads/develop
+srcx search --repo <name> --version <full-commit-A> --query retry
+srcx read --result <result-id> --context 20
+srcx read --result <result-id> --full-file
+srcx read --repo <name> --version <full-commit-A> \
   --path src/example.ts --lines 10:30
 
-node dist/cli.js import --repo <name> --artifact /tmp/srcx-build-b
+srcx import --repo <name> --artifact /tmp/srcx-build-b
 # An old A result still reads A after B is published.
-node dist/cli.js read --result <result-id> --full-file
+srcx read --result <result-id> --full-file
 
-node dist/cli.js git sync-tags --repo <name>
-node dist/cli.js resolve --repo <name> --ref v1.0.0
+srcx git sync-tags --repo <name>
+srcx resolve --repo <name> --ref v1.0.0
 ```
 
 `doctor` checks authentication and Collection read access, not write permission
@@ -314,7 +325,9 @@ identity or Collection name. Registration normalizes the selected Git remote;
 without remotes receive a persistent local source identity. Repeated registration
 preserves existing descriptions/labels and attaches the local checkout.
 
-Search is literal analyzed lexical text, restricted to `kind=chunk`. Optional
+Search defaults to literal analyzed lexical text. All retrieval modes are
+restricted to `kind=chunk`; semantic/hybrid and optional Qwen reranking are
+described above. Optional
 `--path` and `--language` filters are exact keyword filters. `--limit` is 1–100.
 Search results include a durable local `resultId`, commit, immutable Tag/Snapshot,
 path, lines, and citation. Reads verify the original content hash and refuse

package/RELEASING.md

@@ -2,6 +2,18 @@
 
 ## Current state
 
+As of 2026-09-26, the verified `dev` package is `0.1.0-dev.17`, built from
+`98be125d45128a9395544b0925df591c0da08b39` by the successful
+[develop publication](https://github.com/lambdadb/srcx/actions/runs/36178586841).
+The downloaded registry tarball matches its SHA-512 integrity and installed
+`gitHead`. The npm provenance statement identifies the same commit, workflow
+and run; inspecting that statement is separate from cryptographic verification
+of its signature. See [first-use validation](VALIDATION.md#installed-dev-package-first-use)
+for the clean consumer and live CLI checks. `dev` is mutable; these claims apply
+to the exact version above, not future publications.
+
+Bootstrap history:
+
 The initial `@functional-systems/srcx@0.1.0-dev.1` package was published publicly
 from reviewed commit `f94f947bf593b2f8498c07e130933b0ab444f868`. Its exact tarball
 passed clean-install and live LambdaDB checks. The local bootstrap has no GitHub
@@ -12,8 +24,8 @@ The package-specific Trusted Publisher is configured for `lambdadb/srcx` and
 `publish.yaml`, with direct publication allowed. `NPM_DEV_PUBLISH_ENABLED=true`.
 The first successful [OIDC publication](https://github.com/lambdadb/srcx/actions/runs/36035063463/attempts/3)
 produced `0.1.0-dev.3` from the same commit and moved `dev` to that version.
-Provenance identifies this repository, workflow, commit, and run. The current
-`latest` remains the bootstrap prerelease `0.1.0-dev.1`; it is not a stable release.
+Provenance identifies this repository, workflow, commit, and run. At the September 26 registry check,
+`latest` remained the bootstrap prerelease `0.1.0-dev.1`; it is not a stable release.
 See [VALIDATION.md](VALIDATION.md) for registry and consumer verification.
 
 This workflow follows [LambdaDB CLI](https://github.com/lambdadb/lambdadb-cli/blob/develop/RELEASING.md).
@@ -54,7 +66,7 @@ GitHub permissions and no LambdaDB service keys.
 Publication packs once, tests that exact tarball through a clean installation,
 and publishes the same file with provenance. Do not replace it with a repacked
 artifact after testing. The package includes runtime JavaScript, source maps and
-their source files, and public docs/license; it excludes tests, workflows,
+their source files, the optional Qwen worker/requirements, and public docs/license; it excludes tests, workflows,
 scripts, `.env`, and `.srcx` state. Production dependencies supply the parser
 WASM assets; package tests exercise parsing with install scripts disabled.
 

package/VALIDATION.md

@@ -2,6 +2,83 @@
 
 Date: 2026-09-26 (Asia/Seoul).
 
+## Developer workflow pilot
+
+The [workflow pilot](https://github.com/lambdadb/srcx/blob/develop/eval/DEVELOPER-WORKFLOW-PILOT.md)
+used installed dev.17 against srcx `7521974a331471b264a695e6f9795bba3e4a7dfd`
+for three assistant-selected maintenance investigations. Five searches and 16
+source-verified reads produced change maps for Qwen diagnostics, Go chunking and
+automatic Git sync. No feature implementation or comparative quality claim is
+part of this record.
+
+One import added 127 files / 699 chunks to the existing managed-small Collection,
+with 283,104 estimated document embedding input tokens. Two query embedding
+requests and one cached Qwen invocation stayed within predeclared bounds. All
+live commands completed without replay. Two separate local failure probes
+confirmed indistinguishable public errors for missing Python and an invalid
+device; raw exceptions remained masked.
+
+Evidence is retained in the workflow-pilot worktree under `.srcx/workflow-pilot/`:
+
+- `plan.json`: SHA-256 `86d929c19ff69faaed6a264198e0b9c15bf32d3b721a31c503a08c4d2e12da39`.
+- `run.json`: SHA-256 `f0ca1a5e6ac12d48e4929959b85b08e846fb239011125cb153aaa6d0babb1359`.
+- `error-probes.json`: SHA-256 `a5fb5c44dc089466cee8a6bba1c4d374c46bba01f9eef24ac739cfeac72b8612`.
+
+## Installed dev package first use
+
+On 2026-09-26, the public `@functional-systems/srcx@0.1.0-dev.17` tarball from
+source `98be125d45128a9395544b0925df591c0da08b39` passed a fresh npm consumer
+installation and **23 CLI commands** covering local setup and live LambdaDB. The corresponding
+[develop CI and publication](https://github.com/lambdadb/srcx/actions/runs/36178586841)
+completed successfully. The registry `dev` tag selected dev.17 at inspection;
+`latest` still selected the bootstrap dev.1, not a stable release.
+
+The downloaded tarball's SHA-512 matched npm metadata, and its installed package
+version/`gitHead` matched the intended source. Both npm attestation subjects
+matched those tarball bytes; the provenance statement identified the expected
+repository, commit, workflow and run. The attestation signature was not
+independently verified in this local check.
+
+The consumer used `npm install --ignore-scripts` in a new prefix, an isolated
+`SRCX_CONFIG`/`SRCX_STATE_DIR`, and the installed `.bin/srcx` rather than the
+checkout build. The existing Python 3.12 environment and cached Qwen model were
+reused explicitly; all 34 pinned Python package versions and their active
+installed dependency constraints matched. This was not a clean Python/model
+installation, CPU/CUDA validation, or a new ranking benchmark.
+
+| Check                           | Observed result                                                                                                      |
+| ------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
+| Version/help, configure, doctor | Expected dev.17 version/options; fresh connection settings; authentication/read check passed                         |
+| Preview, register, discover     | New synthetic Git repository; lexical preview had two files, two chunks, 44 tokens; one isolated Collection created  |
+| Commit A import/search/read     | Validated publication, matching lexical result and exact original file                                               |
+| Optional local Qwen             | Real cached model, finite reranker score, matching source read; 4.56 s external CLI wall time                        |
+| Git tag mapping                 | `v1` Alias resolved to commit A                                                                                      |
+| Commit B import/search/read     | Changed source visible; deleted-file query returned no hits; A result handle still read exact A source               |
+| Managed semantic/hybrid         | One search/read per mode against an existing immutable Collection of public srcx source; selected source matched Git |
+
+The two synthetic commits were A `0d3bdc318ad1079339a46bc67cd7037f0ebfc3cc`
+and B `9d652332bf1d29545970ccda2d823b8d4d1f627f`. Import commands took 90.45 s
+and 41.44 s, including index visibility and Tag validation. These are single
+integration measurements; they are not import-throughput or latency guarantees.
+
+Usage was **one new lexical Collection, two imports, six searches and two managed
+query embeddings**, with no new document embeddings or model downloads. All
+commands completed; no failed-command replay was needed. Synthetic remote
+resources and local artifacts remain available for inspection.
+
+Retained evidence lives under `.srcx/first-use/` in the first-use worktree:
+
+- `run/report.json`: reservations, command timings, versions and checks;
+  SHA-256 `10652c0cbf485e39be576f37b196873e66b284b6f07d3f2c5a86ebf72a7ff7ea`.
+- `package/functional-systems-srcx-0.1.0-dev.17.tgz`: exact tested registry artifact;
+  SHA-256 `ec75eedd8c9031d5de3cace38f90f84ccf46f9b9723b13cbfc2e73ae9c7edde7`.
+- `registry.json`, `dist-tags.json`, `attestations.json`, `python-runtime.json`,
+  bounded `acceptance.mjs`, command outputs, fixture source and result handles.
+
+The documentation follow-up changes no runtime code, search defaults, published
+package versions or deployment configuration. Later docs-only dev builds are
+separate artifacts; the above live evidence remains attached to dev.17.
+
 ## Local reranker integration
 
 The opt-in `search --rerank qwen` path was exercised with the cached
@@ -503,19 +580,30 @@ Those temporary paths are run-specific and are not committed fixtures.
 
 ## Implementation choices and remaining scope
 
-The initial connected preset is lexical (`embedding=none`). The internal embedding
-interface verifies eligibility/cache behavior only; production provider selection,
-semantic/hybrid querying, pricing, and evaluation remain open. Public imports
-reject a vector-enabled build to prevent fixture vectors from reaching LambdaDB.
-
-Git branch imports reuse a fixed `git-*` Branch with its last validated immutable
-baseline; SHA/tag-only imports retain frozen `work-*` writers. Automatic Git
-observation, failed-workspace recovery, branch rename/deletion handling, Alias
-pruning, and garbage collection are not yet implemented. Failed/candidate
-resources and build artifacts are retained. A single importing host is required;
-the local lock is not a distributed lease. Remote pending ownership prevents
-accidental adoption from fresh local state but is not a distributed compare-and-set
-or lease protocol.
+Implemented CLI behavior:
+
+- Lexical search remains the default. Separate managed Collections support
+  OpenAI `text-embedding-3-small` (1536 dimensions) and `text-embedding-3-large`
+  (3072 dimensions), with explicit semantic and RRF hybrid search. The CLI rejects
+  arbitrary fixture/custom vector presets, not the supported managed presets.
+- Optional local Qwen reranking uses a pinned model revision, bounded candidates,
+  verified source and stable ties. It requires separate Python/model setup;
+  startup and full-command latency are reported above. Jev comparison is deferred.
+- Git branch imports reuse a fixed `git-*` Branch with its last validated immutable
+  baseline; SHA/tag imports retain frozen `work-*` writers. Explicit `--resume`
+  reconciles the same pending import through its journal. Observed Git tags map
+  to Aliases, including multiple names for one published commit.
+- Local diagnostics, public retrieval baselines and the fixed-candidate CoSQA
+  reranking comparison are complete within their documented bounds. They do not
+  establish universal relevance gains, production throughput or agent-task success.
+
+Remaining operational scope: automatic Git observation, recovery of abandoned
+workspaces without their required journal, branch rename/deletion handling,
+authoritative Alias pruning, retention/garbage collection, and configuration
+migration. Failed/candidate resources and local build artifacts are retained.
+A single importing host is required: local locks and remote pending ownership
+are not a distributed compare-and-set or lease protocol. UI, MCP and hosted SaaS
+remain outside the first CLI scope. No stable/rc or Homebrew release is claimed.
 
 Parsing uses the pinned packaged grammars. Unsupported/new syntax can fall back
 and is recorded; no claim of complete language-version coverage is made. There is

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@functional-systems/srcx",
-  "version": "0.1.0-dev.17",
+  "version": "0.1.0-dev.19",
   "private": false,
   "description": "Version-aware code search on LambdaDB",
   "type": "module",
@@ -64,5 +64,5 @@
     "url": "https://github.com/lambdadb/srcx/issues"
   },
   "license": "Apache-2.0",
-  "gitHead": "98be125d45128a9395544b0925df591c0da08b39"
+  "gitHead": "cd7dbc9d4682f7713669ff7c7b98e0481b583c11"
 }