@functional-systems/srcx 0.1.0-dev.4 → 0.1.0-dev.40

package/CHANGELOG.md

@@ -1,5 +1,184 @@
 # Changelog
 
+## Unreleased
+
+- Evaluate experimental segmented artifacts across 50 successive local versions,
+  including cumulative writes, retained inode allocation, ancestor cleanup and
+  cold/explicit-checkpoint recovery. Add bounded multi-generation managed
+  publication regressions for cache/journal loss, stale or incomplete candidates,
+  retries in each phase and immutable historical reads. Document manual local
+  retention; keep runtime behavior and the JSONL default unchanged.
+
+- Add experimental `import --artifact-storage segments` for self-contained,
+  digest-verified local format-2 artifacts. Reuse unchanged per-file objects with
+  hardlinks and explicit copy fallback; retain format-1 reads and the JSONL default,
+  complete source/hash/vector validation, resumable publication and immutable reads.
+  Preview JSON adds `artifactFormat`; existing artifacts reject a storage override.
+  Measure complete build time, writes and additional inode allocation separately
+  from the earlier storage-only prototype; this is not a default promotion.
+
+- Batch local artifact records in a bounded 64 KiB write buffer and hash their
+  existing canonical representation instead of serializing each record twice.
+  Preserve the artifact format, record order/hashes, document limits and full
+  validation; write larger records directly and remove partial output on failure.
+  Add write-count, canonical-byte and failure-cleanup regressions, and document
+  local build profiles separately from database performance.
+
+- Reduce incremental publication reads by comparing against a fully validated,
+  identity-matched local baseline when available, with immutable remote fallback
+  after local file loss. Reuse complete candidate validation within one invocation
+  after checking its Tag/Snapshot identity, but revalidate on publication retries.
+  Skip unchanged inventory-part uploads; preserve full candidate/hash/vector
+  checks, resumability and immutable version reads. Document remaining full-corpus
+  work and cover small updates to a larger corpus with local operation/byte tests.
+
+- Add `lookup --symbol` for exact indexed definition/declaration lookup with
+  verified source chunks and pinned citations in one call. Support path/language
+  filters and a bounded result limit, using existing indexes without embeddings
+  or reindexing; document the workflow in the bundled agent skill.
+
+- Preserve Java source and test packages named `build`, `dist`, `target` or
+  `coverage` under `src/<source-set>/java/`, while still excluding output trees
+  above the source root and nested dependencies/caches. Advance the pinned source
+  policy to `source-v3`; new registrations/builds use a distinct configuration,
+  and presets from the earlier source policy are not accepted by this runtime.
+
+- Document a frozen nine-session comparison of semantic retrieval against adaptive
+  local and lexical search, including cleaned corpora, equal answer coverage,
+  measured overhead and the absence of a demonstrated semantic task advantage.
+
+- Look up explicit generated Collection names directly instead of discovering
+  every repository. Exact Collection names take precedence over aliases; only
+  missing Collections fall back to discovery. Preserve descriptor, schema and
+  immutable-version validation and ambiguity checks for repository names/keys.
+
+- Add optional `read --implementation` for contiguous Python and TypeScript/TSX
+  overload groups. Return the pinned implementation with accurate citations;
+  preserve the selected span when linking is unsupported, ambiguous or too large.
+  Keep indexing and search defaults unchanged.
+
+- Separate file selection from embedding eligibility. Exclude credential and
+  crypto payloads, virtual environments, caches and logs; keep lockfiles, generated
+  code, snapshots, tabular data and legal/author files available lexically without
+  embedding them. Fix case-sensitive minified-asset exclusions. Add pinned
+  `--file-policy` JSON rules for registration and local previews, explicit inventory
+  reasons and evaluation corpus exclusions for `eval/**`.
+
+- Exclude image assets, including text SVG and portable pixel maps, before
+  indexing or embedding. Match image extensions case-insensitively, preserve
+  exclusion metadata, and keep inline SVG in source files searchable. Include
+  the source policy change in preset identity.
+
+- Fix Shell indexing crashes on Bash comparisons such as `[[ "$value" == 1 ]]`
+  by bundling a runtime-compatible, checksum-pinned Bash grammar with its license.
+  Include the grammar in preset identity and installed-package validation.
+
+- Bundle the srcx-search agent skill in npm and add `skills install`, `update`,
+  `remove` and `status` for Codex and Claude Code, with user/project scopes,
+  explicit setup and preservation of customized or unowned files. Removing an
+  absent skill stays read-only even when its parent directory already exists.
+
+- Explain source-language query selection in README and search help, including
+  agent translation of query intent while preserving exact identifiers and
+  answering in the user's language. Keep multilingual input accepted without
+  adding automatic translation or changing retrieval defaults.
+
+- Keep standard text analysis as the default and add `--analyzers` for any combination of
+  LambdaDB's English, Korean, Japanese and standard analyzers. Pin the canonical
+  selection in repository/build identity and verify it during discovery. This
+  pre-release preset requires an analyzer list; older presets are not accepted.
+  Make publication readiness independent of stop-word/tokenization behavior.
+  Pin existing evaluation harnesses to standard regardless of product defaults.
+
+- Add Python/Go/Rust/C/C++/Shell/SQL syntax chunking with language-specific
+  declaration, function and statement boundaries, symbol/scope metadata and
+  bounded source-preserving splits. Bundle the pinned SQL grammar with licenses.
+
+- Distinguish Qwen setup, cache, device, input-size and timeout failures with
+  actionable messages while keeping raw worker diagnostics private.
+
+- Record three developer search/read investigations with concrete change maps,
+  bounded live usage and source verification, separate from ranking benchmarks.
+
+- Refresh implemented/remaining scope and verify the published dev.17 package
+  through a fresh npm consumer and bounded live first-use workflow.
+
+- Add opt-in local Qwen search reranking with bounded candidate pools, pinned
+  source evidence, separate scores and per-command timing.
+
+- Evaluate fixed-candidate CosQA semantic reranking with local Qwen, shared
+  duplicate scores, stable ties and separate official-ID/content metrics.
+
+- Cross-check the retained CosQA lexical baseline with an independent offline
+  BM25 ranker and the same StandardAnalyzer tokenization.
+
+- Add pinned CoIR CoSQA and MTEB CodeSearchNet Python/Go retrieval baselines,
+  sharing srcx query construction and using trec_eval metrics on official labels.
+
+- Add a bounded local Qwen reranking diagnostic with frozen inputs, durable
+  per-pair reservations and offline source-selection reporting.
+
+- Prepare offline reranking inputs from verified retained Click/Cobra candidates,
+  with separate evaluator labels, original ranks and source provenance.
+
+- Add a pending human review worksheet and fixed-candidate reranking experiment plan.
+
+- Run bounded, explicitly unreviewed Click/Cobra retrieval diagnostics with managed small and unchanged source labels.
+
+- Retain allowlisted CLI evaluation failure diagnostics without raw output or credentials.
+- Prepare an offline, unreviewed Click/Cobra question set with pinned source evidence.
+- Default CLI search/read evaluation across two pinned public repositories, with
+  alternative answer sets, exact source verification, and complete stdout accounting.
+- Internal reproducible lexical chunking evaluation with pinned source labels,
+  equal path-only enrichment, isolated live indexes, and token-budget metrics.
+
+### Added
+
+- Opt-in managed `text-embedding-3-large` (3072 dimensions) in separate pinned
+  Collections, with paired small/large evaluation and unchanged small identities.
+
+- Paired identifier/natural-language/mixed-query diagnostics on eight new tasks,
+  with verified candidate-rank coverage and offline stdout-budget analysis.
+- Bounded actual-CLI lexical/semantic/hybrid evaluation on identical managed
+  corpora, with reviewed and original labels, per-question regressions, stdout
+  token counts, command timings and durable request reservations.
+
+- Opt-in LambdaDB managed OpenAI `text-embedding-3-small` Collections, offline
+  embedding input previews, and explicit semantic/RRF hybrid search modes.
+- Candidate validation of generated vectors separately from exact source payloads,
+  unchanged-record reuse, and a bounded synthetic managed acceptance harness.
+
+- Persistent one-to-one Git branch / Collection Branch mapping, branch-local
+  incremental updates, and branch-selected search, resolve, and read against the
+  last published commit Tag.
+- Remote pending import ownership and resumable in-place branch updates, including
+  shared commit Tags across branches and non-fast-forward Git branch changes.
+- `test:live:branches` for repeatable synthetic live acceptance with retained
+  checkpoint, retry, and resource evidence.
+
+### Changed
+
+- Git branch imports update their fixed `git-*` writer; SHA/tag imports retain
+  manual workspaces. Existing publications and pending manual journals remain
+  usable. See README for migration and artifact behavior.
+- Ambiguous indexed branch/Git tag names require explicit `refs/heads/...` or
+  `refs/tags/...` selectors.
+
+### Fixed
+
+- Count failed and unsupported public-benchmark queries separately, retaining
+  both in the ranking metric denominators.
+
+- Fetch omitted managed vectors from query responses through the same immutable
+  Tag, preserving hit order/scores and requiring exact non-vector payload equality.
+
+- Reject an unprepared CLI evaluation run before creating its root/lock so a
+  subsequent offline preparation can use the same path.
+- Pin branch control before enumerating versions so a concurrent publication
+  cannot advance the selector beyond the version list being read.
+- Reject live checkpoint reuse across different code/harness inputs and preserve
+  the original validation revision and completion time on same-input reruns.
+
 ## [0.1.0-dev.1] - 2026-09-25
 
 ### Added

package/CONTRIBUTING.md

@@ -36,6 +36,10 @@ version/help, Git import and WASM parsing, resumable publication against a
 loopback fixture, version-selected search, and exact reads. It does not call
 LambdaDB or publish npm packages.
 
+The Node test runner has a two-minute timeout per test-file process. The CLI
+contract file runs several sequential subprocess-based scenarios, so its combined
+runtime on shared CI runners can exceed one minute even when each scenario passes.
+
 Keep package and lockfile versions synchronized; the executable reads the package
 version. Add changes to `CHANGELOG.md`. Document flag/JSON/exit-code migrations
 before shipping incompatible behavior. During 0.x, use a minor version for