@functional-systems/srcx 0.1.0-dev.7 → 0.1.0-dev.9

package/CHANGELOG.md

@@ -9,6 +9,20 @@
 
 ### Added
 
+- Opt-in managed `text-embedding-3-large` (3072 dimensions) in separate pinned
+  Collections, with paired small/large evaluation and unchanged small identities.
+
+- Paired identifier/natural-language/mixed-query diagnostics on eight new tasks,
+  with verified candidate-rank coverage and offline stdout-budget analysis.
+- Bounded actual-CLI lexical/semantic/hybrid evaluation on identical managed
+  corpora, with reviewed and original labels, per-question regressions, stdout
+  token counts, command timings and durable request reservations.
+
+- Opt-in LambdaDB managed OpenAI `text-embedding-3-small` Collections, offline
+  embedding input previews, and explicit semantic/RRF hybrid search modes.
+- Candidate validation of generated vectors separately from exact source payloads,
+  unchanged-record reuse, and a bounded synthetic managed acceptance harness.
+
 - Persistent one-to-one Git branch / Collection Branch mapping, branch-local
   incremental updates, and branch-selected search, resolve, and read against the
   last published commit Tag.
@@ -27,6 +41,9 @@
 
 ### Fixed
 
+- Fetch omitted managed vectors from query responses through the same immutable
+  Tag, preserving hit order/scores and requiring exact non-vector payload equality.
+
 - Reject an unprepared CLI evaluation run before creating its root/lock so a
   subsequent offline preparation can use the same path.
 - Pin branch control before enumerating versions so a concurrent publication

package/DESIGN.md

@@ -39,7 +39,7 @@ Confirmed boundaries from the discussion:
 - Map each Git tag to a LambdaDB Alias targeting its published commit Tag.
   Multiple Git tags for the same commit share one commit Tag and Snapshot.
 - Keep search results and subsequent source reads pinned to the selected version.
-- Replace a changed file's entire chunk set initially; cache embeddings by input.
+- Replace a changed file's entire chunk set initially; retain unchanged remote records.
 - Keep core LambdaDB work, GitHub App/webhooks, UI, MCP, and a public SaaS outside
   the first deliverable. sbrain reference documents remain unchanged.
 - No remote repository creation, application commit, push, publication, real
@@ -379,9 +379,10 @@ readers on multiple machines do not require local registration state.
    changed/deleted file, compute obsolete IDs as previous file/chunk IDs minus
    replacement IDs. A deleted file has an empty replacement set. Obtain previous
    IDs from the inventory, not mutable query-by-path results.
-6. Apply the pinned per-chunk embedding policy. If embeddings are enabled, batch
-   eligible cache misses across chunks/files within the provider's token/request
-   limits. Validate successful vectors before caching.
+6. Apply the pinned per-chunk embedding policy. The managed preset materializes
+   eligible `embeddingText` inputs without calling a provider locally. LambdaDB
+   generates vectors during ordinary upsert. Skip unchanged remote records and
+   validate generated vectors in the candidate Tag before publication.
    Bound queue memory; do not load an entire repository's content into RAM.
 7. Materialize deterministic records and deletion lists as local build artifacts.
    Produce size-bounded inventory parts and a root manifest template. Check all
@@ -389,8 +390,10 @@ readers on multiple machines do not require local registration state.
    journal supplies the separate attempt ID when sealing the final manifest.
 
 Dry-run executes the same inventory/chunking path but performs no database mutation
-or embedding call. For pending embedding misses, report tokens and projected vector
-size separately; do not claim a completed vector/records hash. Report every entry,
+or embedding call. Report `managed` eligible chunks and `managedTokens` (estimated
+input tokens for the complete corpus), with `embedded=0` in the local artifact.
+These are not billed usage or incremental upload counts. Record hashes bind the
+submitted payload; server-generated vectors are validated separately. Report every entry,
 exclusion reason, parse status, byte size, chunk/token counts, embedding eligibility
 and skip reasons, and change summary. Dry-run plans may show pending embedding work;
 published chunk records may not silently treat missing required vectors as skips.
@@ -473,7 +476,7 @@ fields are omitted when absent; records of different kinds need not share them.
 | Record | Additional stored fields |
 | --- | --- |
 | File (`kind=file`) | `path`, `blobOid`, `contentHash`, `byteLength`, `sourceText`, `language`, `parseStatus`, `chunkCount` |
-| Chunk (`kind=chunk`) | `fileId`, `path`, parent `contentHash`, `language`, `chunkKind`, `startByte`, `endByte`, `startLine`, `endLine`, `ordinal`, `searchText`, `embeddingStatus`; optional `symbol`, `scope`, `signature`, `embeddingSkipReason`, `embeddingInputHash`, `embedding` |
+| Chunk (`kind=chunk`) | `fileId`, `path`, parent `contentHash`, `language`, `chunkKind`, `startByte`, `endByte`, `startLine`, `endLine`, `ordinal`, `searchText`, `embeddingStatus`; optional `symbol`, `scope`, `signature`, `embeddingSkipReason`, `embeddingInputHash`, `embeddingText`, `embedding` |
 | Corpus root (`kind=manifest`, `role=corpus`, ID `__manifest__`) | `repoId`, `indexId`, full `commitOid`, `commitTime`, `requestedRef`, full `config`, `inventoryHash`, `recordsHash`, `counts`, `buildId`, `attemptId`, `inventoryPartIds` |
 | Inventory part (`kind=manifest`, `role=inventory`) | `partOrdinal`, `partHash`, ordered `entries` with path, file/chunk IDs, blob/content identities, inclusion/exclusion reasons, parse states, sizes, and embedding coverage |
 | Repository control (`kind=manifest`, `role=repository`, main ID `__repo__`) | Full source identity, repo/index IDs, immutable preset/config, initialization state |
@@ -487,8 +490,13 @@ content such as function, declaration, documentation, imports, or fallback; it i
 separate from the file/chunk/manifest discriminator. `counts` includes included and
 excluded files, chunks, and embedded/skipped chunks with reason totals.
 
-In a completed corpus, `embeddingStatus=embedded` requires a valid vector and
-`embeddingInputHash`, and omits the skip reason. `embeddingStatus=skipped` requires
+The managed preset uses `embeddingStatus=managed` to identify its generation policy,
+not local completion. Eligible chunks carry `embeddingText=searchText` and an
+`embeddingInputHash` over the exact input and embedding configuration; ordinary
+upserts omit `embedding`. A completed candidate must contain a nonzero, finite
+1536-number vector for every eligible chunk. Local artifacts contain no vectors.
+The internal fixture-only client embedder retains `embeddingStatus=embedded`;
+its arbitrary presets cannot be published through the CLI. `embeddingStatus=skipped` requires
 an explicit policy reason and omits the vector and input hash. The lexical-only
 preset records `embedding-disabled`; an embedding-enabled preset records its
 specific exclusion reason. Pending or failed required embedding work stays in the
@@ -549,10 +557,13 @@ baseline, not a proven code-specific ranking solution. An initial enrichment
 candidate preserves original identifiers while adding camelCase/snake_case terms;
 pin and evaluate that transformation before adopting it as the preset.
 
-An embedding-enabled preset adds an unmanaged `embedding` vector field with a
-concrete model-selected dimension and similarity metric, fixed before Collection
-creation. An indexed vector field does not require a vector on every chunk; file
-and manifest records also omit it. The `none` preset omits the vector index entirely.
+The opt-in managed preset adds indexed text `embeddingText` and a managed
+`embedding` vector field with provider `openai`, model `text-embedding-3-small`,
+1536 dimensions and cosine similarity. The source field, dimensions and similarity
+are nested under the vector field's `embedding` configuration; there are no
+top-level dimension/similarity properties. Collection discovery verifies the
+complete resolved schema and pinned preset. Missing `embeddingText` on skipped
+chunks, files and manifests means no vector is generated for those records. The `none` preset omits the vector index entirely.
 Changing presets creates a separate index; do not silently mutate an existing
 serving index. Do not write placeholder or test vectors into a real corpus.
 
@@ -741,13 +752,36 @@ choice and the finer skip rules remain evaluation decisions, not measured result
 
 ### Embedding reuse
 
-Cache embeddings by the exact full input and provider/model/revision/dimensions/
-preprocessing settings. Do not include commit, line numbers, file IDs, or build IDs
-in that input. A line shift can update positions while reusing the same vector;
-changed chunk boundaries or a renamed path in the input can legitimately miss.
-Cache only successful validated outputs atomically. Query embeddings use the same
-compatible contract and have their own cache/budget accounting. Provider aliases
-may drift if no immutable model revision exists; record that limitation.
+The managed implementation reuses unchanged records from the immutable baseline:
+compare all submitted payload fields after separately validating and removing only
+the server-generated vector. Do not upload those unchanged records again. Direct
+managed vector writes are forbidden, including copies fetched from an old Tag.
+Changed files receive new chunk IDs and may require re-embedding even if some text
+is identical. Replaying an uncertain write may also incur embedding usage. There
+is no srcx-managed document or query vector cache and no promised server cache.
+The older local cache remains only for internal client-embedder fixtures.
+
+The input includes path/symbol context and source text, not commit/line/build IDs.
+The preset pins provider/model/dimensions/similarity and input policy, but the
+provider model name is not an immutable model revision. Validation proves vector
+shape and payload integrity, not deterministic numerical reproduction by OpenAI.
+
+Search remains lexical by default. Explicit `--mode semantic` uses managed
+`knn.queryText`; `--mode hybrid` combines the existing lexical query and that kNN
+query using LambdaDB RRF. Both legs enforce `kind=chunk` and the same optional
+path/language filters; kNN uses a prefilter and `k=limit`. The lexical leg keeps
+nonembedded chunks eligible. Query/fetch/list request vectors explicitly so the
+same validation applies to pinned result handles. When a deployment omits managed
+vectors from list responses, fetch the affected IDs from that same immutable Tag
+and require identical non-vector payloads before vector validation. Source reads still use the
+original file bytes, never enriched text. No relevance gain is claimed yet.
+
+Managed source text and semantic/hybrid query text are sent through LambdaDB to
+OpenAI and incur embedding usage plus normal LambdaDB operations/storage. Dry-run
+and lexical queries do not request query embeddings. Use explicit opt-in and a
+bounded synthetic acceptance run before evaluating private or larger corpora.
+See [managed embeddings](https://docs.lambdadb.ai/guides/collections/managed-embeddings)
+and [hybrid query](https://docs.lambdadb.ai/guides/search/hybrid) for the API contract.
 
 ## 9. Candidate validation, publication, and `versions`
 
@@ -1005,7 +1039,7 @@ Required meaningful checks:
 - Byte coverage, Unicode/BOM/CRLF/trailing-newline/long-line spans are exact.
 - Parse failures and huge nodes stay bounded and visible in reports.
 - B contains additions/replacements/deletions while A reads remain unchanged.
-- A controlled line-shift fixture updates positions and reuses identical embeddings.
+- A controlled line-shift fixture updates positions; managed imports may re-embed changed IDs, while the internal client-cache fixture verifies input-based reuse.
 - Different paths remain distinct even with identical bytes; reuse follows input.
 - Fixed-config mismatches fail before writes; fake vectors never enter a real corpus.
 - Explicit obsolete-ID deletion preserves replacement IDs and removes surplus chunks.

package/README.md

@@ -21,7 +21,7 @@ srcx --help
 
 ## Run locally
 
-Requires Node.js 22.14+ and Git. The initial preset uses lexical search with
+Requires Node.js 22.14+ and Git. The default preset uses lexical search with
 `embedding=none`; no embedding account or paid API is needed.
 
 ```sh
@@ -77,6 +77,59 @@ Chunks cover the exact source bytes, carry one-based line ranges, target 800 tok
 and stay below 1,500 tokens including path/symbol context. The internal window
 baseline is available to the test/build API, not as a public storage backend.
 
+## Opt into managed embeddings
+
+Supported models are `text-embedding-3-small` (1536 dimensions) and
+`text-embedding-3-large` (3072 dimensions), both cosine. Use either name with
+`--embedding`; each has a separate pinned preset and Collection. Existing small
+Collections keep their identity. Changing models requires a separate import.
+
+LambdaDB generates vectors for meaningful code, tests and prose. Imports-only
+and structural chunks remain lexical. The CLI uses LambdaDB credentials; it does not need an OpenAI key.
+Source text sent for embedding and semantic/hybrid queries pass through LambdaDB
+to OpenAI and incur usage charges. See [LambdaDB managed embeddings](https://docs.lambdadb.ai/guides/collections/managed-embeddings).
+
+```sh
+# Offline preview: no upload or embedding request.
+srcx import --path /path/to/repo --ref main --dry-run \
+  --embedding text-embedding-3-small --output /tmp/srcx-managed-preview
+
+# Creates a separate Collection; existing lexical Collections stay usable.
+srcx repo add --path /path/to/repo --embedding text-embedding-3-small
+# Use the exact collection value returned above, especially with multiple presets.
+srcx import --repo <collection> --ref main
+srcx search --repo <collection> --version main --query "retry failed writes" --mode hybrid
+srcx search --repo <collection> --version main --query "retry failed writes" --mode semantic
+```
+
+Connected imports, including `--dry-run --repo`, use the selected repository's
+pinned preset. `--embedding` on import is only for `--dry-run --path`.
+`search` keeps `--mode lexical` as its default; semantic and hybrid require a
+managed Collection. Hybrid uses LambdaDB RRF with identical filters on both legs.
+Existing result handles, lexical config hashes and publication journals stay valid.
+
+Local artifacts contain eligible `embeddingText` and input hashes, not vectors.
+`counts.managed` and `counts.managedTokens` estimate the complete corpus's eligible
+chunks and input tokens; `embedded=0` does not mean the server will skip them.
+Publication requires valid generated vectors in the immutable candidate Tag,
+plus the same exact payload/source checks as lexical publication. Unchanged
+records are not uploaded again. Changed file chunks and uncertain write retries
+may be embedded again; no global or query embedding cache is promised.
+
+A small synthetic live check is available after building:
+
+```sh
+npm run build
+node --env-file=.env.local scripts/live-managed.mjs
+```
+
+It uploads two synthetic commits, bounds document inputs to 10,000 estimated
+tokens and issues four managed search queries plus lexical checks. It retains a
+Collection, local fixture, artifacts, result handles and `.srcx/live-managed/report.json`.
+Completed reruns preserve the original report and make no service calls; changed
+runtime inputs require a fresh directory. Failed runs retain journals and reject
+automatic replay. These are correctness checks, not retrieval-quality measurements.
+
 ## Evaluate retrieval quality
 
 The internal [lexical chunking pilot](https://github.com/lambdadb/srcx/blob/develop/eval/README.md) compares syntax-aware and
@@ -89,6 +142,22 @@ adds a second public repository and verifies real `search` → `read` results wi
 the unchanged CLI preset. It uses normal repository Collections and separate local
 state; see its documented effects before opting into the live run.
 
+The [retrieval mode comparison](https://github.com/lambdadb/srcx/blob/develop/eval/RETRIEVAL-MODES.md) runs lexical, semantic and
+hybrid against the same managed corpora. It retains original and reviewed labels,
+full stdout accounting, command timings and bounded embedding request estimates.
+`npm run eval:modes:prepare` is offline; live execution is explicit and incurs
+managed embedding and LambdaDB usage.
+
+The [query-style diagnostic](https://github.com/lambdadb/srcx/blob/develop/eval/QUERY-STYLES.md)
+pairs identifiers, natural-language descriptions and mixed questions on eight new
+tasks. It adds rank-prefix and stdout-budget measurements without changing the
+model or ranking. [Embedding model candidates](https://github.com/lambdadb/srcx/blob/develop/eval/EMBEDDING-MODELS.md)
+separately describe possible follow-up comparisons and integration requirements.
+The [managed model comparison](https://github.com/lambdadb/srcx/blob/develop/eval/MODEL-COMPARISON.md)
+compares small and large on both frozen suites, including lexical controls.
+[Results](https://github.com/lambdadb/srcx/blob/develop/eval/MODEL-COMPARISON-RESULTS.md)
+show mixed gains and regressions; small remains the initial managed choice.
+
 ## Explicit live acceptance run
 
 The live harness accepts `LAMBDADB_BASE_URL`, `LAMBDADB_PROJECT_NAME`, and

package/VALIDATION.md

@@ -12,6 +12,124 @@ concurrency, or retrieval-quality benchmark. npm publication evidence is recorde
 separately below. The persistent Git branch changes have a separate synthetic
 live run recorded next; the earlier live runs do not validate this new path.
 
+## Retrieval mode comparison
+
+At application/harness commit `de5a23bbc2f0712d134066354b4313420d21fccd`, the
+actual CLI completed 48 searches across the same two managed corpora and fixed
+16-question suite. All **456 results/handles** and **236 reads** matched pinned
+source and version facts. Revised-label complete evidence was lexical **16/16**,
+semantic **8/16**, and hybrid **13/16**; unchanged original labels scored 14/16,
+8/16, and 11/16. These are diagnostic coverage measurements, not human answer grades.
+
+All three hybrid regressions contained the required evidence in search's top ten,
+but outside the first five reads. Keep lexical as the default pending fresh-task
+ranking evaluation. The run also exposed a query response omitting one managed
+vector; same-Tag fetch hydration now preserves non-vector payload equality and
+hit order/scores. The failed attempt remains separate from the complete fresh run.
+
+Node 22/24 passed **67 tests**, typechecking and installed-package CLI checks after
+the fix. The completed live report was revalidated without new service calls, and
+its bytes/timestamp were unchanged. See [full results](eval/RETRIEVAL-MODE-RESULTS.md)
+for per-mode metrics, inspected misses, input/usage accounting, source revisions,
+report hashes and evidence limits; [protocol](eval/RETRIEVAL-MODES.md) documents
+reproduction and effects.
+
+## Query-style supplement
+
+At harness commit `65c8e7060366f4df846c9b2bc08270e7337f0df1`, 72 searches tested
+eight new tasks with identifier, natural-language and mixed formulations using
+the same managed small model and the same two immutable publications. All
+**696 result handles and 356 reads** matched source/version facts. Complete
+first-five evidence by lexical/semantic/hybrid was **7/8, 7/8, 7/8** for identifiers,
+**0/8, 3/8, 1/8** for natural descriptions and **5/8, 7/8, 6/8** for mixed queries.
+These are eight paired tasks, not 24 independent judgments. Semantic helped on
+this diagnostic; the original keyword-heavy result must not be generalized.
+
+The frozen protocol adds verified candidate coverage at ranks 1/3/5/10, the first
+complete prefix, stdout tokens and offline token-budget prefixes. No model,
+ranking, read-selection or product default changed. The original suite/results
+and incomplete run remain intact. A completed rerun preserved report bytes and
+timestamp without service calls. Node 22.14.0 and 24.15.0 passed **69 tests**;
+typecheck, formatting and frozen-harness CI passed. See [results and cases](eval/QUERY-STYLE-RESULTS.md),
+[protocol](eval/QUERY-STYLES.md), and [embedding model research](eval/EMBEDDING-MODELS.md).
+
+## Managed small versus large comparison
+
+At frozen harness `6fa65b541dc50debdfa4f9a6297301b2cb3c2b53`, small (1536 dimensions)
+and large (3072 dimensions) completed 240 searches over identical pinned source/chunk inputs.
+All **2,304 result handles and 1,184 reads** passed source/version validation.
+Style-suite complete top-five evidence was semantic **17/24 → 16/24** and hybrid
+**14/24 → 16/24**; natural semantic improved **3/8 → 4/8**. The separate regression
+set scored semantic **8/16 → 8/16** and hybrid **13/16 → 12/16**. All 40 lexical
+candidate lists matched across models. Small coverage repeated the earlier runs.
+
+An initial large attempt stopped twice with an unknown CLI search error; its
+19 partial rows and two successful diagnostic queries remain excluded. A fresh
+isolated run reused the same publications and completed without changing the
+frozen protocol. All four completed reports revalidated without service calls
+and retained their bytes/timestamps. Keep small as the initial managed choice
+and large opt-in; these familiar diagnostic tasks do not establish a model winner.
+
+Node 22.14.0 and 24.15.0 passed **72 tests**, typecheck and installed-package CLI
+checks covering lexical/small/large. Formatting/version checks and frozen-code
+CI passed. [Full results](eval/MODEL-COMPARISON-RESULTS.md) record per-query changes,
+report hashes, original-label scores, all five roots' reserved usage, and limits;
+[protocol](eval/MODEL-COMPARISON.md) records the predeclared comparison.
+
+Review follow-up: resume and completed replay now reject saved rows whose category
+is missing or differs from the frozen question, even when the stored summary was
+recomputed to match the corruption. The subprocess regression first reproduced
+the acceptance bug, then passed with the guard; valid replay and rejected reports
+both preserve their bytes. Node 24.15.0 passed **73 tests** after this change.
+The four completed model reports have no category mismatches and remain unchanged.
+Their frozen harness remains the revision recorded above; the new harness requires
+fresh run roots rather than rewriting historical runtime fingerprints.
+
+## Managed OpenAI embedding acceptance
+
+The opt-in managed preset passed a separate synthetic live run at application and
+harness commit `177731386f76a13723a83aa8470d4e0715e29424` using Node 24.15.0 and
+the user-supplied development connection. It completed at
+`2026-09-24T20:46:46.305Z` in 111.706 seconds. Only synthetic source was uploaded.
+
+- Both commits published validated immutable Tags, with six generated 1536-number
+  vectors in each corpus. Mixed file/manifest records correctly omitted vectors.
+- Six filtered searches (lexical, semantic and hybrid for each version) returned
+  14 result handles; every full-file read matched the pinned Git blob byte for byte.
+- The second import skipped unchanged records, removed deleted content, and kept
+  the original version's handle readable. Fresh repository discovery preserved the
+  managed preset and schema.
+- Ordinary upserts submitted ten eligible chunk inputs across both imports. The
+  preflight upper bound was 167 estimated document input tokens; four searches
+  requested query embeddings. These counts are not provider-reported billed usage.
+
+The first run at `fb5b6d8` correctly remained unpublished: this deployment omitted
+vectors from list responses despite `includeVectors=true`, while fetch/query on
+that same Tag returned them. The transport now fetches those listed IDs from the
+same immutable Tag and requires exact non-vector payload equality before vector
+validation. Regression checks reject missing fetch records and changed payloads.
+The failed run and retry journal remain under `.srcx/live-managed/`; they were not
+relabelled as successful evidence. The successful run has a separate Collection
+and evidence directory, `.srcx/live-managed-verified/`, in the validation worktree.
+Both synthetic Collections are retained for inspection.
+
+The successful report SHA-256 is
+`6c96ac8af9b79c85e98c84f1f01851906f0b1b3e4d96b51d9417cb59380ac532`;
+its runtime fingerprint is
+`7d8a8006820ee523a88e80d6614327000ce305cffafe377d18fda5b50ba9c947`.
+A same-runtime rerun made no service calls and preserved the original report bytes.
+The harness rejects changed runtime/destination/Node inputs and incomplete runs
+before new requests; recovery after uncertain writes remains explicit.
+
+Node 22.14.0 and 24.15.0 each passed **61 tests**, typechecking, and installed-tarball
+CLI checks for both presets. Formatting, version validation and diff checks passed.
+Local tests cover generated-vector failures, exact payload validation, unchanged
+record reuse, uncertain-write recovery, preset/schema drift, query serialization,
+and credential-free managed previews. Live checks used the built application APIs;
+installed-package CLI checks used the loopback fault-injection store.
+These checks establish integration/source correctness, not semantic relevance gains
+or performance on real repositories. The earlier lexical evaluations remain separate.
+
 ## Default CLI search/read evaluation
 
 The follow-up uses actual built CLI subprocesses with the unchanged default

package/dist/build.d.ts

@@ -32,9 +32,76 @@ export type Preset = {
     embedding: {
         model: string;
         dimensions: number;
+        managed?: true;
+        provider?: "openai";
+        similarity?: "cosine";
+        sourceField?: "embeddingText";
     } | null;
 };
 export declare const PRESET: Preset;
+/** Opt-in managed preset; PRESET stays byte-for-byte compatible with lexical corpora. */
+export declare const MANAGED_PRESET: Preset;
+export declare const MANAGED_LARGE_PRESET: Preset;
+export declare function presetFor(embedding?: string): Preset;
+export declare function supportedPreset(preset: unknown): preset is Preset;
+export declare function indexConfigs(preset?: Preset): {
+    readonly kind: {
+        readonly type: "keyword";
+    };
+    readonly path: {
+        readonly type: "keyword";
+    };
+    readonly language: {
+        readonly type: "keyword";
+    };
+    readonly fileId: {
+        readonly type: "keyword";
+    };
+    readonly symbol: {
+        readonly type: "keyword";
+    };
+    readonly searchText: {
+        readonly type: "text";
+        readonly analyzers: readonly ["standard"];
+    };
+} | {
+    embeddingText: {
+        type: string;
+        analyzers: string[];
+    };
+    embedding: {
+        type: string;
+        managedEmbedding: boolean;
+        embedding: {
+            model: string;
+            dimensions: number;
+            provider?: "openai";
+            similarity?: "cosine";
+            sourceField?: "embeddingText";
+        };
+    };
+    kind: {
+        readonly type: "keyword";
+    };
+    path: {
+        readonly type: "keyword";
+    };
+    language: {
+        readonly type: "keyword";
+    };
+    fileId: {
+        readonly type: "keyword";
+    };
+    symbol: {
+        readonly type: "keyword";
+    };
+    searchText: {
+        readonly type: "text";
+        readonly analyzers: readonly ["standard"];
+    };
+};
+/** Validate server enrichment separately; all other payload fields remain exact. */
+export declare function recordHash(doc: Doc, preset: Preset): string;
 export type InventoryItem = {
     path: string;
     pathBase64: string;
@@ -50,6 +117,8 @@ export type InventoryItem = {
     parseStatus?: string;
     tokens?: number;
     embedded?: number;
+    managed?: number;
+    managedTokens?: number;
     skipped?: number;
 };
 export type Build = {

package/dist/build.js

@@ -20,6 +20,67 @@ export const PRESET = {
     maxFileBytes: 1024 * 1024,
     embedding: null,
 };
+/** Opt-in managed preset; PRESET stays byte-for-byte compatible with lexical corpora. */
+export const MANAGED_PRESET = {
+    ...PRESET,
+    embedding: {
+        managed: true,
+        provider: "openai",
+        model: "text-embedding-3-small",
+        dimensions: 1536,
+        similarity: "cosine",
+        sourceField: "embeddingText",
+    },
+};
+export const MANAGED_LARGE_PRESET = {
+    ...MANAGED_PRESET,
+    embedding: {
+        ...MANAGED_PRESET.embedding,
+        model: "text-embedding-3-large",
+        dimensions: 3072,
+    },
+};
+export function presetFor(embedding = "none") {
+    invariant(["none", "text-embedding-3-small", "text-embedding-3-large"].includes(embedding), "Embedding must be none, text-embedding-3-small or text-embedding-3-large.");
+    if (embedding === "none")
+        return PRESET;
+    return embedding === "text-embedding-3-large"
+        ? MANAGED_LARGE_PRESET
+        : MANAGED_PRESET;
+}
+export function supportedPreset(preset) {
+    return (preset !== undefined &&
+        [hash(PRESET), hash(MANAGED_PRESET), hash(MANAGED_LARGE_PRESET)].includes(hash(preset)));
+}
+export function indexConfigs(preset = PRESET) {
+    if (!preset.embedding?.managed)
+        return INDEX_CONFIGS;
+    const { managed: _, ...embedding } = preset.embedding;
+    return {
+        ...INDEX_CONFIGS,
+        embeddingText: { type: "text", analyzers: ["standard"] },
+        embedding: { type: "vector", managedEmbedding: true, embedding },
+    };
+}
+/** Validate server enrichment separately; all other payload fields remain exact. */
+export function recordHash(doc, preset) {
+    if (!preset.embedding?.managed)
+        return hash(doc);
+    const { embedding, ...source } = doc;
+    if (doc.kind === "chunk" && doc.embeddingStatus === "managed") {
+        invariant(typeof doc.embeddingText === "string" &&
+            doc.embeddingText === doc.searchText &&
+            doc.embeddingInputHash ===
+                hash([preset.embedding, doc.embeddingText]) &&
+            Array.isArray(embedding) &&
+            embedding.length === preset.embedding.dimensions &&
+            embedding.every((v) => typeof v === "number" && Number.isFinite(v)) &&
+            embedding.some((v) => v !== 0), "Missing or invalid managed embedding.");
+    }
+    else
+        invariant(embedding === undefined && doc.embeddingText === undefined, "Unexpected managed embedding on an ineligible record.");
+    return hash(source);
+}
 export async function* records(directory) {
     const input = createReadStream(join(directory, "records.jsonl"), {
         encoding: "utf8",
@@ -62,6 +123,7 @@ export async function materialize(args) {
         await validateBuild(prev);
     }
     invariant(!preset.embedding ||
+        (preset.embedding.managed && supportedPreset(preset)) ||
         (args.embedder?.model === preset.embedding.model &&
             args.embedder.dimensions === preset.embedding.dimensions), "Embedding provider must match the pinned preset.");
     const commit = args.resolvedCommit ?? (await resolveCommit(args.identity.path, args.ref));
@@ -192,26 +254,36 @@ export async function materialize(args) {
                 const eligible = !["imports", "structural"].includes(span.chunkKind);
                 if (preset.embedding && eligible) {
                     const inputHash = hash([preset.embedding, span.searchText]);
-                    const cachePath = args.cache
-                        ? join(args.cache, inputHash + ".json")
-                        : undefined;
-                    let vector = cachePath
-                        ? await optionalJson(cachePath)
-                        : undefined;
-                    if (!vector) {
-                        vector = await args.embedder.embed(span.searchText);
+                    if (preset.embedding.managed) {
+                        doc.embeddingText = span.searchText;
+                        doc.embeddingInputHash = inputHash;
+                        doc.embeddingStatus = "managed";
+                        delete doc.embeddingSkipReason;
+                        item.managed = (item.managed ?? 0) + 1;
+                        item.managedTokens = (item.managedTokens ?? 0) + span.tokenCount;
+                    }
+                    else {
+                        const cachePath = args.cache
+                            ? join(args.cache, inputHash + ".json")
+                            : undefined;
+                        let vector = cachePath
+                            ? await optionalJson(cachePath)
+                            : undefined;
+                        if (!vector) {
+                            vector = await args.embedder.embed(span.searchText);
+                            invariant(vector.length === preset.embedding.dimensions &&
+                                vector.every(Number.isFinite), "Invalid embedding output.");
+                            if (cachePath)
+                                await atomic(cachePath, vector);
+                        }
                         invariant(vector.length === preset.embedding.dimensions &&
-                            vector.every(Number.isFinite), "Invalid embedding output.");
-                        if (cachePath)
-                            await atomic(cachePath, vector);
+                            vector.every(Number.isFinite), "Invalid cached vector.");
+                        doc.embedding = vector;
+                        doc.embeddingInputHash = inputHash;
+                        doc.embeddingStatus = "embedded";
+                        delete doc.embeddingSkipReason;
+                        item.embedded++;
                     }
-                    invariant(vector.length === preset.embedding.dimensions &&
-                        vector.every(Number.isFinite), "Invalid cached vector.");
-                    doc.embedding = vector;
-                    doc.embeddingInputHash = inputHash;
-                    doc.embeddingStatus = "embedded";
-                    delete doc.embeddingSkipReason;
-                    item.embedded++;
                 }
                 else {
                     doc.embeddingSkipReason = preset.embedding
@@ -289,6 +361,12 @@ export async function materialize(args) {
             excluded: items.filter((e) => e.status === "excluded").length,
             chunks: items.reduce((n, e) => n + (e.chunkIds?.length ?? 0), 0),
             tokens: items.reduce((n, e) => n + (e.tokens ?? 0), 0),
+            ...(preset.embedding?.managed
+                ? {
+                    managed: items.reduce((n, e) => n + (e.managed ?? 0), 0),
+                    managedTokens: items.reduce((n, e) => n + (e.managedTokens ?? 0), 0),
+                }
+                : {}),
             embedded: items.reduce((n, e) => n + (e.embedded ?? 0), 0),
             skipped: items.reduce((n, e) => n + (e.skipped ?? 0), 0),
         },
@@ -322,6 +400,11 @@ export async function validateBuild(b) {
         invariant(!seen.has(doc.id) && b.recordHashes[doc.id] === hash(doc), "Corrupted or unexpected artifact record.");
         seen.add(doc.id);
         invariant(doc.configHash === b.configHash && doc.schemaVersion === 1, "Mixed configuration.");
+        if (b.preset.embedding?.managed) {
+            invariant(doc.embedding === undefined, "Managed artifacts must not contain vectors.");
+            if (doc.kind !== "chunk")
+                invariant(doc.embeddingText === undefined, "Only eligible chunks can contain embedding input.");
+        }
         if (doc.kind === "file") {
             finish();
             activeFile = doc;
@@ -362,7 +445,23 @@ export async function validateBuild(b) {
                 doc.searchText.endsWith(slice.toString("utf8")) &&
                 doc.tokenCount === tokens(doc.searchText) &&
                 Number(doc.tokenCount) <= CHUNKER.maxTokens, "Invalid chunk source/position/token binding.");
-            if (doc.embeddingStatus === "embedded")
+            if (b.preset.embedding?.managed) {
+                const eligible = !["imports", "structural"].includes(String(doc.chunkKind));
+                invariant(eligible
+                    ? doc.embeddingStatus === "managed" &&
+                        doc.embeddingText === doc.searchText &&
+                        doc.embeddingInputHash ===
+                            hash([b.preset.embedding, doc.searchText]) &&
+                        doc.embeddingSkipReason === undefined
+                    : doc.embeddingStatus === "skipped" &&
+                        doc.embeddingText === undefined &&
+                        doc.embeddingInputHash === undefined &&
+                        doc.embeddingSkipReason ===
+                            (doc.chunkKind === "imports"
+                                ? "imports-only"
+                                : "structural-only"), "Invalid managed embedding input or eligibility.");
+            }
+            else if (doc.embeddingStatus === "embedded")
                 invariant(b.preset.embedding &&
                     Array.isArray(doc.embedding) &&
                     doc.embedding.length === b.preset.embedding.dimensions &&