@functional-systems/srcx 0.1.0-dev.1 → 0.1.0-dev.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,59 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ - Retain allowlisted CLI evaluation failure diagnostics without raw output or credentials.
6
+ - Prepare an offline, unreviewed Click/Cobra question set with pinned source evidence.
7
+ - Default CLI search/read evaluation across two pinned public repositories, with
8
+ alternative answer sets, exact source verification, and complete stdout accounting.
9
+ - Internal reproducible lexical chunking evaluation with pinned source labels,
10
+ equal path-only enrichment, isolated live indexes, and token-budget metrics.
11
+
12
+ ### Added
13
+
14
+ - Opt-in managed `text-embedding-3-large` (3072 dimensions) in separate pinned
15
+ Collections, with paired small/large evaluation and unchanged small identities.
16
+
17
+ - Paired identifier/natural-language/mixed-query diagnostics on eight new tasks,
18
+ with verified candidate-rank coverage and offline stdout-budget analysis.
19
+ - Bounded actual-CLI lexical/semantic/hybrid evaluation on identical managed
20
+ corpora, with reviewed and original labels, per-question regressions, stdout
21
+ token counts, command timings and durable request reservations.
22
+
23
+ - Opt-in LambdaDB managed OpenAI `text-embedding-3-small` Collections, offline
24
+ embedding input previews, and explicit semantic/RRF hybrid search modes.
25
+ - Candidate validation of generated vectors separately from exact source payloads,
26
+ unchanged-record reuse, and a bounded synthetic managed acceptance harness.
27
+
28
+ - Persistent one-to-one Git branch / Collection Branch mapping, branch-local
29
+ incremental updates, and branch-selected search, resolve, and read against the
30
+ last published commit Tag.
31
+ - Remote pending import ownership and resumable in-place branch updates, including
32
+ shared commit Tags across branches and non-fast-forward Git branch changes.
33
+ - `test:live:branches` for repeatable synthetic live acceptance with retained
34
+ checkpoint, retry, and resource evidence.
35
+
36
+ ### Changed
37
+
38
+ - Git branch imports update their fixed `git-*` writer; SHA/tag imports retain
39
+ manual workspaces. Existing publications and pending manual journals remain
40
+ usable. See README for migration and artifact behavior.
41
+ - Ambiguous indexed branch/Git tag names require explicit `refs/heads/...` or
42
+ `refs/tags/...` selectors.
43
+
44
+ ### Fixed
45
+
46
+ - Fetch omitted managed vectors from query responses through the same immutable
47
+ Tag, preserving hit order/scores and requiring exact non-vector payload equality.
48
+
49
+ - Reject an unprepared CLI evaluation run before creating its root/lock so a
50
+ subsequent offline preparation can use the same path.
51
+ - Pin branch control before enumerating versions so a concurrent publication
52
+ cannot advance the selector beyond the version list being read.
53
+ - Reject live checkpoint reuse across different code/harness inputs and preserve
54
+ the original validation revision and completion time on same-input reruns.
55
+
56
+ ## [0.1.0-dev.1] - 2026-09-25
57
+
5
58
  ### Added
6
59
 
7
60
  - Git commit imports, syntax-aware chunks, immutable LambdaDB versions, lexical
@@ -17,4 +70,6 @@
17
70
  - Pending imports resume without the previous local build artifact.
18
71
  - Search/read `--version` selects the corpus instead of printing the CLI version.
19
72
 
20
- No npm release has been published yet.
73
+ Initial npm bootstrap published to the `dev` channel from commit
74
+ `f94f947bf593b2f8498c07e130933b0ab444f868`. This local bootstrap has no
75
+ GitHub Actions provenance. No stable release has been published.
package/DESIGN.md CHANGED
@@ -18,14 +18,13 @@ supporting each result. The project name is `srcx`.
18
18
  | --- | --- |
19
19
  | Project and display name | `srcx` |
20
20
  | Local directory | `srcx` |
21
- | Intended GitHub repository | `lambdadb/srcx` |
21
+ | GitHub repository | `lambdadb/srcx` |
22
22
  | CLI executable | `srcx` |
23
- | Intended npm package | `@functional-systems/srcx` |
23
+ | npm package | `@functional-systems/srcx` |
24
24
  | Short description | Version-aware code search on LambdaDB |
25
25
 
26
- The name is selected; the GitHub repository and npm package have not been created
27
- or published in this project. The package manifest and executable entry point are
28
- present in the initial CLI. Keep display/configuration naming centralized and
26
+ The public GitHub repository and npm development channel are available. Current
27
+ publication evidence and stable-release status are recorded in RELEASING.md. Keep display/configuration naming centralized and
29
28
  distribution identity in `package.json`. Database IDs and the existing
30
29
  `code-*` Collection naming and `purpose=code-search-v1` protocol marker remain
31
30
  independent of the product name.
@@ -40,7 +39,7 @@ Confirmed boundaries from the discussion:
40
39
  - Map each Git tag to a LambdaDB Alias targeting its published commit Tag.
41
40
  Multiple Git tags for the same commit share one commit Tag and Snapshot.
42
41
  - Keep search results and subsequent source reads pinned to the selected version.
43
- - Replace a changed file's entire chunk set initially; cache embeddings by input.
42
+ - Replace a changed file's entire chunk set initially; retain unchanged remote records.
44
43
  - Keep core LambdaDB work, GitHub App/webhooks, UI, MCP, and a public SaaS outside
45
44
  the first deliverable. sbrain reference documents remain unchanged.
46
45
  - No remote repository creation, application commit, push, publication, real
@@ -72,9 +71,10 @@ This document is not a deployed-revision audit or evidence of a passing live run
72
71
 
73
72
  The initial command set below is implemented; run it as `node dist/cli.js` from
74
73
  this checkout after `npm run build`. See README for exact options and recovery.
75
- The first implementation uses fresh `work-*` Branches and observed-local-tag Alias
76
- synchronization; persistent tracked branches and authoritative pruning remain
77
- follow-up work. Immutable manifest `requestedRef` is normalized to the resolved
74
+ Git branch imports now reuse persistent `git-*` Branches; explicit commit/tag
75
+ imports use frozen `work-*` writers. Observed-local-tag Alias synchronization is
76
+ implemented; automatic Git observation and authoritative pruning remain follow-up
77
+ work. Immutable manifest `requestedRef` is normalized to the resolved
78
78
  commit OID; the
79
79
  original user spelling remains in the local build artifact. The broader contract
80
80
  below remains the target design.
@@ -379,9 +379,10 @@ readers on multiple machines do not require local registration state.
379
379
  changed/deleted file, compute obsolete IDs as previous file/chunk IDs minus
380
380
  replacement IDs. A deleted file has an empty replacement set. Obtain previous
381
381
  IDs from the inventory, not mutable query-by-path results.
382
- 6. Apply the pinned per-chunk embedding policy. If embeddings are enabled, batch
383
- eligible cache misses across chunks/files within the provider's token/request
384
- limits. Validate successful vectors before caching.
382
+ 6. Apply the pinned per-chunk embedding policy. The managed preset materializes
383
+ eligible `embeddingText` inputs without calling a provider locally. LambdaDB
384
+ generates vectors during ordinary upsert. Skip unchanged remote records and
385
+ validate generated vectors in the candidate Tag before publication.
385
386
  Bound queue memory; do not load an entire repository's content into RAM.
386
387
  7. Materialize deterministic records and deletion lists as local build artifacts.
387
388
  Produce size-bounded inventory parts and a root manifest template. Check all
@@ -389,8 +390,10 @@ readers on multiple machines do not require local registration state.
389
390
  journal supplies the separate attempt ID when sealing the final manifest.
390
391
 
391
392
  Dry-run executes the same inventory/chunking path but performs no database mutation
392
- or embedding call. For pending embedding misses, report tokens and projected vector
393
- size separately; do not claim a completed vector/records hash. Report every entry,
393
+ or embedding call. Report `managed` eligible chunks and `managedTokens` (estimated
394
+ input tokens for the complete corpus), with `embedded=0` in the local artifact.
395
+ These are not billed usage or incremental upload counts. Record hashes bind the
396
+ submitted payload; server-generated vectors are validated separately. Report every entry,
394
397
  exclusion reason, parse status, byte size, chunk/token counts, embedding eligibility
395
398
  and skip reasons, and change summary. Dry-run plans may show pending embedding work;
396
399
  published chunk records may not silently treat missing required vectors as skips.
@@ -473,11 +476,12 @@ fields are omitted when absent; records of different kinds need not share them.
473
476
  | Record | Additional stored fields |
474
477
  | --- | --- |
475
478
  | File (`kind=file`) | `path`, `blobOid`, `contentHash`, `byteLength`, `sourceText`, `language`, `parseStatus`, `chunkCount` |
476
- | Chunk (`kind=chunk`) | `fileId`, `path`, parent `contentHash`, `language`, `chunkKind`, `startByte`, `endByte`, `startLine`, `endLine`, `ordinal`, `searchText`, `embeddingStatus`; optional `symbol`, `scope`, `signature`, `embeddingSkipReason`, `embeddingInputHash`, `embedding` |
479
+ | Chunk (`kind=chunk`) | `fileId`, `path`, parent `contentHash`, `language`, `chunkKind`, `startByte`, `endByte`, `startLine`, `endLine`, `ordinal`, `searchText`, `embeddingStatus`; optional `symbol`, `scope`, `signature`, `embeddingSkipReason`, `embeddingInputHash`, `embeddingText`, `embedding` |
477
480
  | Corpus root (`kind=manifest`, `role=corpus`, ID `__manifest__`) | `repoId`, `indexId`, full `commitOid`, `commitTime`, `requestedRef`, full `config`, `inventoryHash`, `recordsHash`, `counts`, `buildId`, `attemptId`, `inventoryPartIds` |
478
481
  | Inventory part (`kind=manifest`, `role=inventory`) | `partOrdinal`, `partHash`, ordered `entries` with path, file/chunk IDs, blob/content identities, inclusion/exclusion reasons, parse states, sizes, and embedding coverage |
479
482
  | Repository control (`kind=manifest`, `role=repository`, main ID `__repo__`) | Full source identity, repo/index IDs, immutable preset/config, initialization state |
480
- | Ref control (`kind=manifest`, `role=git-ref`, main) | Full Git ref name, encoded Alias/Branch name, observed commit/time, applied version and synchronization state |
483
+ | Git tag control (`kind=manifest`, `role=git-ref`, main) | Full Git tag ref, encoded Alias name, observed commit, applied version and synchronization state |
484
+ | Git branch control (`kind=manifest`, `role=git-branch`, main) | Full Git branch ref, encoded Branch name, validated writer baseline, canonical published Tag and pending attempt identity |
481
485
  | Validation summary (`kind=manifest`, `role=validation`, main) | Candidate name/Snapshot, build/config/attempt identity, verifier version and checked counts/hashes; no premature claim of publication |
482
486
 
483
487
  `startByte`/`endByte` are half-open; display lines are one-based/inclusive.
@@ -486,8 +490,13 @@ content such as function, declaration, documentation, imports, or fallback; it i
486
490
  separate from the file/chunk/manifest discriminator. `counts` includes included and
487
491
  excluded files, chunks, and embedded/skipped chunks with reason totals.
488
492
 
489
- In a completed corpus, `embeddingStatus=embedded` requires a valid vector and
490
- `embeddingInputHash`, and omits the skip reason. `embeddingStatus=skipped` requires
493
+ The managed preset uses `embeddingStatus=managed` to identify its generation policy,
494
+ not local completion. Eligible chunks carry `embeddingText=searchText` and an
495
+ `embeddingInputHash` over the exact input and embedding configuration; ordinary
496
+ upserts omit `embedding`. A completed candidate must contain a nonzero, finite
497
+ 1536-number vector for every eligible chunk. Local artifacts contain no vectors.
498
+ The internal fixture-only client embedder retains `embeddingStatus=embedded`;
499
+ its arbitrary presets cannot be published through the CLI. `embeddingStatus=skipped` requires
491
500
  an explicit policy reason and omits the vector and input hash. The lexical-only
492
501
  preset records `embedding-disabled`; an embedding-enabled preset records its
493
502
  specific exclusion reason. Pending or failed required embedding work stays in the
@@ -548,10 +557,13 @@ baseline, not a proven code-specific ranking solution. An initial enrichment
548
557
  candidate preserves original identifiers while adding camelCase/snake_case terms;
549
558
  pin and evaluate that transformation before adopting it as the preset.
550
559
 
551
- An embedding-enabled preset adds an unmanaged `embedding` vector field with a
552
- concrete model-selected dimension and similarity metric, fixed before Collection
553
- creation. An indexed vector field does not require a vector on every chunk; file
554
- and manifest records also omit it. The `none` preset omits the vector index entirely.
560
+ The opt-in managed preset adds indexed text `embeddingText` and a managed
561
+ `embedding` vector field with provider `openai`, model `text-embedding-3-small`,
562
+ 1536 dimensions and cosine similarity. The source field, dimensions and similarity
563
+ are nested under the vector field's `embedding` configuration; there are no
564
+ top-level dimension/similarity properties. Collection discovery verifies the
565
+ complete resolved schema and pinned preset. Missing `embeddingText` on skipped
566
+ chunks, files and manifests means no vector is generated for those records. The `none` preset omits the vector index entirely.
555
567
  Changing presets creates a separate index; do not silently mutate an existing
556
568
  serving index. Do not write placeholder or test vectors into a real corpus.
557
569
 
@@ -740,13 +752,36 @@ choice and the finer skip rules remain evaluation decisions, not measured result
740
752
 
741
753
  ### Embedding reuse
742
754
 
743
- Cache embeddings by the exact full input and provider/model/revision/dimensions/
744
- preprocessing settings. Do not include commit, line numbers, file IDs, or build IDs
745
- in that input. A line shift can update positions while reusing the same vector;
746
- changed chunk boundaries or a renamed path in the input can legitimately miss.
747
- Cache only successful validated outputs atomically. Query embeddings use the same
748
- compatible contract and have their own cache/budget accounting. Provider aliases
749
- may drift if no immutable model revision exists; record that limitation.
755
+ The managed implementation reuses unchanged records from the immutable baseline:
756
+ compare all submitted payload fields after separately validating and removing only
757
+ the server-generated vector. Do not upload those unchanged records again. Direct
758
+ managed vector writes are forbidden, including copies fetched from an old Tag.
759
+ Changed files receive new chunk IDs and may require re-embedding even if some text
760
+ is identical. Replaying an uncertain write may also incur embedding usage. There
761
+ is no srcx-managed document or query vector cache and no promised server cache.
762
+ The older local cache remains only for internal client-embedder fixtures.
763
+
764
+ The input includes path/symbol context and source text, not commit/line/build IDs.
765
+ The preset pins provider/model/dimensions/similarity and input policy, but the
766
+ provider model name is not an immutable model revision. Validation proves vector
767
+ shape and payload integrity, not deterministic numerical reproduction by OpenAI.
768
+
769
+ Search remains lexical by default. Explicit `--mode semantic` uses managed
770
+ `knn.queryText`; `--mode hybrid` combines the existing lexical query and that kNN
771
+ query using LambdaDB RRF. Both legs enforce `kind=chunk` and the same optional
772
+ path/language filters; kNN uses a prefilter and `k=limit`. The lexical leg keeps
773
+ nonembedded chunks eligible. Query/fetch/list request vectors explicitly so the
774
+ same validation applies to pinned result handles. When a deployment omits managed
775
+ vectors from list responses, fetch the affected IDs from that same immutable Tag
776
+ and require identical non-vector payloads before vector validation. Source reads still use the
777
+ original file bytes, never enriched text. No relevance gain is claimed yet.
778
+
779
+ Managed source text and semantic/hybrid query text are sent through LambdaDB to
780
+ OpenAI and incur embedding usage plus normal LambdaDB operations/storage. Dry-run
781
+ and lexical queries do not request query embeddings. Use explicit opt-in and a
782
+ bounded synthetic acceptance run before evaluating private or larger corpora.
783
+ See [managed embeddings](https://docs.lambdadb.ai/guides/collections/managed-embeddings)
784
+ and [hybrid query](https://docs.lambdadb.ai/guides/search/hybrid) for the API contract.
750
785
 
751
786
  ## 9. Candidate validation, publication, and `versions`
752
787
 
@@ -814,10 +849,16 @@ they do not authorize the next commit on that workspace.
814
849
  attempt identity and verifier version. Use an ID derived from the full commit
815
850
  OID and candidate Snapshot ID so attempts cannot overwrite another Snapshot's
816
851
  evidence. Confirm its readback before publication.
817
- 5. Create the deterministic published Tag **from the validated candidate Tag**,
818
- not from the mutable Branch. Verify both Tags have the same Snapshot ID and
819
- matching manifest. Publication changes naming, not corpus content.
820
- 6. Update optional branch-progress/control records and return the published version.
852
+ 5. For a first publication, create the deterministic published Tag **from the
853
+ validated candidate Tag**, not from the mutable Branch. Verify both Tags have
854
+ the same Snapshot ID and matching manifest. If the commit is already published,
855
+ validate and reuse its canonical Tag; retain this writer's validated candidate
856
+ separately when its Snapshot ID differs. Publication never overwrites a commit
857
+ Tag with another writer's snapshot.
858
+ 6. For a tracked Git branch, update its control record with the validated writer
859
+ snapshot and published commit Tag, clear pending ownership, and return the
860
+ published version. Branch selectors keep using the preceding published Tag
861
+ until this control update succeeds.
821
862
  A delayed bookkeeping write does not erase an already published immutable Tag.
822
863
 
823
864
  An empty Snapshot or incomplete candidate never becomes published. A stale Tag
@@ -827,6 +868,33 @@ avoid repeatedly validating known stale Snapshot IDs. A payload/build defect
827
868
  requires explicit same-build recovery, not endless polling. Publication must
828
869
  finish before the same code Branch starts a later commit.
829
870
 
871
+ ### Persistent Git branch state
872
+
873
+ Each full `refs/heads/...` name maps to one `git-<ref-digest>` writer and one
874
+ `branch-<full-ref-digest>` control document on main (`role=git-branch`). The control
875
+ record holds the last applied immutable baseline, canonical published Tag name,
876
+ and pending attempt/build/commit identity. Record pending ownership before branch
877
+ creation or corpus writes. A missing local journal cannot authorize a second
878
+ import over pending remote state, even if the committed head still matches the
879
+ old baseline. Only the same journal may resume; automatic abandoned-attempt
880
+ recovery remains unimplemented.
881
+
882
+ Compare the writer head with the recorded baseline snapshot before the next
883
+ import, then reconcile old IDs and hashes through its immutable Tag. This works
884
+ without the prior local artifact and supports a branch moving backward or across
885
+ history. Every target commit still comes from one full committed Git tree.
886
+
887
+ An already published commit does not skip updating a different tracked branch.
888
+ Validate that branch's own candidate, reuse the existing canonical commit Tag,
889
+ and retain the candidate as the writer's exact baseline when the snapshot IDs
890
+ differ. Never overwrite the canonical commit publication with another writer's
891
+ snapshot. This also applies when a branch returns to a previously indexed commit.
892
+
893
+ Branch search/resolve/read selects the last published Tag from the control record,
894
+ including while the next import is pending. This promises the last indexed state,
895
+ not the current Git tip. A first pending import has no searchable branch version.
896
+ An ambiguous tracked branch/Git tag short name requires an explicit heads/tags ref.
897
+
830
898
  ### `versions` reads LambdaDB
831
899
 
832
900
  1. Resolve the repository Collection through remote metadata.
@@ -971,7 +1039,7 @@ Required meaningful checks:
971
1039
  - Byte coverage, Unicode/BOM/CRLF/trailing-newline/long-line spans are exact.
972
1040
  - Parse failures and huge nodes stay bounded and visible in reports.
973
1041
  - B contains additions/replacements/deletions while A reads remain unchanged.
974
- - A controlled line-shift fixture updates positions and reuses identical embeddings.
1042
+ - A controlled line-shift fixture updates positions; managed imports may re-embed changed IDs, while the internal client-cache fixture verifies input-based reuse.
975
1043
  - Different paths remain distinct even with identical bytes; reuse follows input.
976
1044
  - Fixed-config mismatches fail before writes; fake vectors never enter a real corpus.
977
1045
  - Explicit obsolete-ID deletion preserves replacement IDs and removes surplus chunks.
package/README.md CHANGED
@@ -4,13 +4,24 @@ Version-aware code search on LambdaDB. Import a committed Git tree, search its
4
4
  chunks, and read exact source pinned to a validated immutable version.
5
5
 
6
6
  This is an experimental development CLI. Local fixture, fault-injection, SDK transport,
7
- and opt-in live acceptance checks are available. The npm package is prepared for
8
- publication but remains unpublished. See [VALIDATION.md](VALIDATION.md) for the latest observed results and
7
+ and opt-in live acceptance checks are available. Development builds are available
8
+ on npm under `@functional-systems/srcx@dev`. See [VALIDATION.md](VALIDATION.md) for the latest observed results and
9
9
  remaining scope, and [DESIGN.md](DESIGN.md) for the full design.
10
10
 
11
+ ## Install a development build
12
+
13
+ Requires Node.js 22.14+ and Git. These are experimental prereleases; there is no
14
+ stable release yet. Select the `dev` channel explicitly:
15
+
16
+ ```sh
17
+ npm install -g @functional-systems/srcx@dev
18
+ srcx --version
19
+ srcx --help
20
+ ```
21
+
11
22
  ## Run locally
12
23
 
13
- Requires Node.js 22.14+ and Git. The initial preset uses lexical search with
24
+ Requires Node.js 22.14+ and Git. The default preset uses lexical search with
14
25
  `embedding=none`; no embedding account or paid API is needed.
15
26
 
16
27
  ```sh
@@ -27,11 +38,11 @@ artifacts. It does not contact LambdaDB. The model's lexical matching is a test
27
38
  convenience, not a Lucene implementation or a relevance benchmark.
28
39
 
29
40
  All commands return JSON, except help and errors. Use `node dist/cli.js` in this
30
- checkout; `srcx` is the configured executable name for future distribution.
41
+ checkout, or `srcx` after installing the npm package.
31
42
 
32
43
  For development branches, CI, and installed-package checks, see
33
44
  [CONTRIBUTING.md](CONTRIBUTING.md). Version channels, first-publication steps, and
34
- the initially disabled npm automation are documented in [RELEASING.md](RELEASING.md).
45
+ npm automation are documented in [RELEASING.md](RELEASING.md).
35
46
  The project is licensed under [Apache-2.0](LICENSE).
36
47
 
37
48
  ## Preview a committed version without credentials
@@ -66,6 +77,92 @@ Chunks cover the exact source bytes, carry one-based line ranges, target 800 tok
66
77
  and stay below 1,500 tokens including path/symbol context. The internal window
67
78
  baseline is available to the test/build API, not as a public storage backend.
68
79
 
80
+ ## Opt into managed embeddings
81
+
82
+ Supported models are `text-embedding-3-small` (1536 dimensions) and
83
+ `text-embedding-3-large` (3072 dimensions), both cosine. Use either name with
84
+ `--embedding`; each has a separate pinned preset and Collection. Existing small
85
+ Collections keep their identity. Changing models requires a separate import.
86
+
87
+ LambdaDB generates vectors for meaningful code, tests and prose. Imports-only
88
+ and structural chunks remain lexical. The CLI uses LambdaDB credentials; it does not need an OpenAI key.
89
+ Source text sent for embedding and semantic/hybrid queries pass through LambdaDB
90
+ to OpenAI and incur usage charges. See [LambdaDB managed embeddings](https://docs.lambdadb.ai/guides/collections/managed-embeddings).
91
+
92
+ ```sh
93
+ # Offline preview: no upload or embedding request.
94
+ srcx import --path /path/to/repo --ref main --dry-run \
95
+ --embedding text-embedding-3-small --output /tmp/srcx-managed-preview
96
+
97
+ # Creates a separate Collection; existing lexical Collections stay usable.
98
+ srcx repo add --path /path/to/repo --embedding text-embedding-3-small
99
+ # Use the exact collection value returned above, especially with multiple presets.
100
+ srcx import --repo <collection> --ref main
101
+ srcx search --repo <collection> --version main --query "retry failed writes" --mode hybrid
102
+ srcx search --repo <collection> --version main --query "retry failed writes" --mode semantic
103
+ ```
104
+
105
+ Connected imports, including `--dry-run --repo`, use the selected repository's
106
+ pinned preset. `--embedding` on import is only for `--dry-run --path`.
107
+ `search` keeps `--mode lexical` as its default; semantic and hybrid require a
108
+ managed Collection. Hybrid uses LambdaDB RRF with identical filters on both legs.
109
+ Existing result handles, lexical config hashes and publication journals stay valid.
110
+
111
+ Local artifacts contain eligible `embeddingText` and input hashes, not vectors.
112
+ `counts.managed` and `counts.managedTokens` estimate the complete corpus's eligible
113
+ chunks and input tokens; `embedded=0` does not mean the server will skip them.
114
+ Publication requires valid generated vectors in the immutable candidate Tag,
115
+ plus the same exact payload/source checks as lexical publication. Unchanged
116
+ records are not uploaded again. Changed file chunks and uncertain write retries
117
+ may be embedded again; no global or query embedding cache is promised.
118
+
119
+ A small synthetic live check is available after building:
120
+
121
+ ```sh
122
+ npm run build
123
+ node --env-file=.env.local scripts/live-managed.mjs
124
+ ```
125
+
126
+ It uploads two synthetic commits, bounds document inputs to 10,000 estimated
127
+ tokens and issues four managed search queries plus lexical checks. It retains a
128
+ Collection, local fixture, artifacts, result handles and `.srcx/live-managed/report.json`.
129
+ Completed reruns preserve the original report and make no service calls; changed
130
+ runtime inputs require a fresh directory. Failed runs retain journals and reject
131
+ automatic replay. These are correctness checks, not retrieval-quality measurements.
132
+
133
+ ## Evaluate retrieval quality
134
+
135
+ The internal [lexical chunking pilot](https://github.com/lambdadb/srcx/blob/develop/eval/README.md) compares syntax-aware and
136
+ window chunks on a fixed public corpus and 16 pinned investigation queries.
137
+ `npm run eval:prepare` builds and validates the comparison offline. Live evaluation
138
+ is an explicit separate operation and retains isolated evaluation Collections.
139
+
140
+ The [default CLI workflow evaluation](https://github.com/lambdadb/srcx/blob/develop/eval/CLI-WORKFLOW.md)
141
+ adds a second public repository and verifies real `search` → `read` results with
142
+ the unchanged CLI preset. It uses normal repository Collections and separate local
143
+ state; see its documented effects before opting into the live run.
144
+
145
+ The [retrieval mode comparison](https://github.com/lambdadb/srcx/blob/develop/eval/RETRIEVAL-MODES.md) runs lexical, semantic and
146
+ hybrid against the same managed corpora. It retains original and reviewed labels,
147
+ full stdout accounting, command timings and bounded embedding request estimates.
148
+ `npm run eval:modes:prepare` is offline; live execution is explicit and incurs
149
+ managed embedding and LambdaDB usage.
150
+
151
+ The [query-style diagnostic](https://github.com/lambdadb/srcx/blob/develop/eval/QUERY-STYLES.md)
152
+ pairs identifiers, natural-language descriptions and mixed questions on eight new
153
+ tasks. It adds rank-prefix and stdout-budget measurements without changing the
154
+ model or ranking. [Embedding model candidates](https://github.com/lambdadb/srcx/blob/develop/eval/EMBEDDING-MODELS.md)
155
+ separately describe possible follow-up comparisons and integration requirements.
156
+ The [managed model comparison](https://github.com/lambdadb/srcx/blob/develop/eval/MODEL-COMPARISON.md)
157
+ compares small and large on both frozen suites, including lexical controls.
158
+ [Results](https://github.com/lambdadb/srcx/blob/develop/eval/MODEL-COMPARISON-RESULTS.md)
159
+ show mixed gains and regressions; small remains the initial managed choice.
160
+
161
+ [Transfer evaluation preparation](https://github.com/lambdadb/srcx/blob/develop/eval/TRANSFER-EVAL.md) adds source-verified,
162
+ unreviewed Python/Go questions; no live retrieval has been run on them.
163
+ New evaluation roots retain [safe command failure diagnostics](https://github.com/lambdadb/srcx/blob/develop/eval/COMMAND-DIAGNOSTICS.md)
164
+ for explicit recovery.
165
+
69
166
  ## Explicit live acceptance run
70
167
 
71
168
  The live harness accepts `LAMBDADB_BASE_URL`, `LAMBDADB_PROJECT_NAME`, and
@@ -85,6 +182,25 @@ It saves connection settings under `.srcx/live/config.json` and keeps its state
85
182
  separate under `.srcx/live/state`. Rerunning resumes the saved synthetic run and
86
183
  refuses to reuse it against a different destination.
87
184
 
185
+ For persistent Git branch acceptance, run `npm run test:live:branches`. It uses a
186
+ separate synthetic Collection and `.srcx/live-branches/` state/report, covering
187
+ in-place A/B updates, last-published reads during a pending update, shared commit
188
+ Tags across two branches, rewinds, no-op imports, and SHA-only manual imports.
189
+ From an isolated worktree, load the original checkout's env file explicitly:
190
+
191
+ ```sh
192
+ npm run build
193
+ node --env-file=/absolute/path/to/srcx/.env.local scripts/live-branches.mjs
194
+ ```
195
+
196
+ The branch harness retains resources and resumes saved checkpoints on rerun.
197
+ It preserves the original source revision and completion time. Checkpoints from
198
+ different runtime code, harness, fixture, lockfile, or Node version are rejected
199
+ before connecting or overwriting evidence. To validate changed code, select a new
200
+ `SRCX_LIVE_BRANCHES_DIR` (for example `.srcx/live-branches-next`); retain the old
201
+ run and any pending journal. Its per-import deadline defaults to 300 seconds and accepts
202
+ `SRCX_LIVE_TIMEOUT_MS` up to 600000. Do not delete pending journals to start over.
203
+
88
204
  ## Connected workflow
89
205
 
90
206
  These commands create and write LambdaDB resources. Select an approved project
@@ -104,6 +220,11 @@ node dist/cli.js repo show --repo <name>
104
220
  # Import the exact artifact reviewed in the credential-free preview.
105
221
  node dist/cli.js import --repo <name> --artifact /tmp/srcx-build-a
106
222
  node dist/cli.js versions --repo <name>
223
+
224
+ # Keep a Git branch connected to its Collection Branch.
225
+ node dist/cli.js import --repo <name> --ref develop
226
+ node dist/cli.js search --repo <name> --version develop --query retry
227
+ node dist/cli.js resolve --repo <name> --ref refs/heads/develop
107
228
  node dist/cli.js search --repo <name> --version <full-commit-A> --query retry
108
229
  node dist/cli.js read --result <result-id> --context 20
109
230
  node dist/cli.js read --result <result-id> --full-file
@@ -141,10 +262,28 @@ are one-based and inclusive.
141
262
  ## Publication and recovery
142
263
 
143
264
  The control Branch `main` holds descriptors/version summaries. `checkpoint-empty`
144
- is forked before any control write. Each import uses a new `work-*` Branch,
145
- optionally forked from the last locally retained, remotely validated writer
146
- baseline. Writers are frozen after publication. A fresh local state can start a
147
- full import from `checkpoint-empty`; source data is never forked from a Tag.
265
+ is forked before any control write. Importing a Git branch creates or updates its
266
+ stable `git-*` Collection Branch, derived from the full `refs/heads/...` name.
267
+ Git `main` also uses a `git-*` Branch; Collection `main` remains control-only.
268
+ Each later import reconciles against that branch's last validated snapshot,
269
+ including deletions and non-fast-forward changes. Branches sharing a commit share
270
+ one canonical `ver-*` Tag while retaining their own writer snapshots.
271
+
272
+ `search/read --version develop` and `resolve --ref develop` select the branch's
273
+ last successfully published commit Tag. During an import or after a failed update,
274
+ they keep serving the previous published commit; the first import is unavailable
275
+ until publication. This is the last indexed state, not a live Git HEAD lookup.
276
+ Nothing polls or fetches Git automatically: run import again to advance the branch.
277
+ If an indexed branch and synchronized Git tag share a short name, use
278
+ `refs/heads/<name>` or `refs/tags/<name>` explicitly. Known Git names take precedence
279
+ over commit prefixes.
280
+
281
+ A commit SHA or Git tag import does not infer a containing branch. An unpublished
282
+ commit uses a fresh `work-*` Branch, optionally forked from a validated previous
283
+ manual writer; an already published commit reuses its Tag. Manual writers remain
284
+ frozen. Source data is never forked from a Tag. A clean tracked branch can be
285
+ updated from fresh local state using its remote baseline. Local build reuse is
286
+ cached separately for each Git branch and for manual imports.
148
287
 
149
288
  The importer deletes obsolete IDs, upserts changed records and inventory parts in
150
289
  bounded sequential requests, then sends `__manifest__` **alone** after every prior
@@ -156,7 +295,10 @@ an early/stale snapshot even after the marker is observed.
156
295
 
157
296
  A per-Collection local lock and durable `pending.json` journal prevent another
158
297
  commit from overwriting a partial attempt. Only one importing host is supported.
159
- After a timeout or unknown write result, retain the artifacts and journal:
298
+ A tracked branch also records pending import ownership in control `main` before
299
+ writing code. If the local journal is lost, new local state refuses to overwrite
300
+ that pending branch even when its committed head has not advanced yet. After a
301
+ timeout or unknown write result, retain the artifacts and journal:
160
302
 
161
303
  ```sh
162
304
  node dist/cli.js import --repo <name> --artifact /path/from/pending.json \
@@ -184,8 +326,17 @@ For search and resolve, a synchronized Git tag name takes precedence over a comm
184
326
  prefix, matching Git import's tag-before-OID rule. A pending tag fails explicitly
185
327
  instead of falling back to a matching commit prefix. Use `refs/tags/<name>` to
186
328
  select a Git tag explicitly.
187
- Explicit authoritative pruning and persistent tracked-branch synchronization are
188
- follow-up work; no full history import is required.
329
+ Automatic Git observation, authoritative pruning, branch rename/deletion handling,
330
+ and garbage collection remain follow-up work; no full history import is required.
331
+
332
+ Existing commit Tags and frozen `work-*` Branches remain valid. The next import
333
+ using a Git branch name establishes its fixed mapping, even if that commit was
334
+ already published. A pending journal from the previous implementation resumes on
335
+ its original `work-*` writer; import the branch again after recovery to establish
336
+ tracking. `--artifact` uses the Git branch recorded when the artifact was built,
337
+ with its pinned commit; it does not reread the branch's current tip. The publication
338
+ JSON continues to describe the canonical commit Tag: its `writer` can be the
339
+ original publishing branch when multiple Git branches share that commit.
189
340
 
190
341
  ## State and implementation map
191
342
 
package/RELEASING.md CHANGED
@@ -2,12 +2,19 @@
2
2
 
3
3
  ## Current state
4
4
 
5
- The source is prepared as `@functional-systems/srcx@0.1.0-dev.1` under Apache-2.0.
6
- It has not been published to npm. `private: false` permits a reviewed first
7
- publication; it does not establish registry availability or distribution readiness.
8
- `NPM_DEV_PUBLISH_ENABLED` is initially `false`. No stable/rc release or Homebrew
9
- formula has been published. Local/live application evidence is in
10
- [VALIDATION.md](VALIDATION.md); it is separate from package publication evidence.
5
+ The initial `@functional-systems/srcx@0.1.0-dev.1` package was published publicly
6
+ from reviewed commit `f94f947bf593b2f8498c07e130933b0ab444f868`. Its exact tarball
7
+ passed clean-install and live LambdaDB checks. The local bootstrap has no GitHub
8
+ Actions provenance. npm created both `dev` and `latest` at that prerelease; select
9
+ `@dev` explicitly. No stable/rc release or Homebrew formula has been published.
10
+
11
+ The package-specific Trusted Publisher is configured for `lambdadb/srcx` and
12
+ `publish.yaml`, with direct publication allowed. `NPM_DEV_PUBLISH_ENABLED=true`.
13
+ The first successful [OIDC publication](https://github.com/lambdadb/srcx/actions/runs/36035063463/attempts/3)
14
+ produced `0.1.0-dev.3` from the same commit and moved `dev` to that version.
15
+ Provenance identifies this repository, workflow, commit, and run. The current
16
+ `latest` remains the bootstrap prerelease `0.1.0-dev.1`; it is not a stable release.
17
+ See [VALIDATION.md](VALIDATION.md) for registry and consumer verification.
11
18
 
12
19
  This workflow follows [LambdaDB CLI](https://github.com/lambdadb/lambdadb-cli/blob/develop/RELEASING.md).
13
20
  The repository keeps Prettier plus strict TypeScript checks as its existing lint
@@ -26,7 +33,8 @@ version entries in `package-lock.json` together, for example using
26
33
  `npm version VERSION --no-git-tag-version`. CLI `--version` reads package metadata.
27
34
  For explicit releases, add a dated `## [VERSION] - YYYY-MM-DD` changelog section.
28
35
  The GitHub tag must match the version, and the prerelease flag must match its channel.
29
- `latest`, not the tip of `main`, identifies the stable npm release.
36
+ After the first stable release, `latest` identifies the stable npm release;
37
+ the bootstrap exception is recorded above.
30
38
 
31
39
  The checked-in development base starts at `0.1.0-dev.1`. During an eligible push,
32
40
  `scripts/dev-release.mjs` replaces `N` with the full history's first-parent commit
@@ -64,7 +72,8 @@ Do not republish, overwrite versions, or move release tags to work around lag.
64
72
 
65
73
  ## First publication and Trusted Publisher setup
66
74
 
67
- Complete these steps from a reviewed, validated commit after this setup PR merges:
75
+ The initial bootstrap is complete. The procedure below documents first-time setup
76
+ and the checks required before declaring automatic publication operational:
68
77
 
69
78
  1. Verify npm ownership for the `functional-systems` scope and the package name.
70
79
  GitHub organization membership does not establish npm permissions.