@ponraaj/dimple-sdk 0.0.0-alpha.6 → 0.0.0-alpha.8

package/README.md

@@ -1,35 +1,18 @@
 # @ponraaj/dimple-sdk
 
-A local-first memory system for AI agents. Store text memories (and whole
-codebases), retrieve them with hybrid semantic + lexical search, and let a
-**self-organizing topic tree + artifact graph + concepts** keep everything
-structured as it grows. Everything runs locally: ONNX embeddings on your
+A local-first memory system for AI agents: store text memories and whole
+codebases, retrieve them with hybrid semantic + lexical search, and let a
+self-organizing **topic tree + artifact graph + concepts** keep everything
+structured as it grows. Everything runs locally — ONNX embeddings on your
 CPU, SQLite files on disk.
 
-This is the programmatic SDK — the
-[`@ponraaj/dimple`](https://www.npmjs.com/package/@ponraaj/dimple) CLI is a
-thin facade over it.
+> **Documentation**: [dimple.ponraaj.me](https://dimple.ponraaj.me) —
+> quickstart, architecture, config reference, changelog ·
+> **Package**: [npmx.dev/package/@ponraaj/dimple](https://npmx.dev/package/@ponraaj/dimple) ·
+> **Status**: alpha (`0.0.0-alpha.n`) — the API is stabilizing.
 
-> **Status**: alpha (`0.0.0-alpha.n`). The API is stabilizing but may change.
-
-### The one-liner
-
-```ts
-import { makeDimple } from "@ponraaj/dimple-sdk"; // ← makeDimple, not dimple
-
-const dimple = await makeDimple({ config: "./dimple.jsonc" });
-const hits = await dimple.query("what did we decide about X", {
-  limit: 5,
-  contentType: "text/plain", // ← code-vs-message separation
-}); // query(text, options) — text is the FIRST positional argument
-await dimple.dispose();
-```
-
-Calling conventions: every method takes **plain arguments** (`query(text,
-opts)`, `write(input, opts)`, `index({ paths })`), every failure is a
-rejected promise with a closed `_tag` (`"Sdk.NotFound"`, `"Sdk.InputError"`,
-…), and `dispose()` is **required** before `process.exit(0)` — the durable
-worker keeps the event loop alive (`dispose()` stops it; it is idempotent).
+This is the programmatic SDK. The [`@ponraaj/dimple`](https://www.npmjs.com/package/@ponraaj/dimple)
+CLI is a thin facade over it.
 
 ## Install
 
@@ -37,16 +20,15 @@ worker keeps the event loop alive (`dispose()` stops it; it is idempotent).
 npm install @ponraaj/dimple-sdk
 ```
 
-Runtime dependencies are the only things installed (Effect, libSQL,
-Transformers.js, drizzle) — the workspace internals (`@dimple/*`) are
-bundled into the package, including fully **self-contained type
-declarations** (the publish build vendors every internal d.ts and rewrites
-the `@dimple/*` specifiers — plain consumers never need the workspace).
+Runtime deps only (Effect, libSQL, transformers) — workspace internals are
+bundled, types are self-contained, and the default install ships **zero
+postinstall scripts** (the local ONNX embedder is an optional peer,
+auto-installed on first use).
 
 ## Quickstart
 
 ```ts
-import { makeDimple } from "@ponraaj/dimple-sdk";
+import { makeDimple } from "@ponraaj/dimple-sdk"; // ← makeDimple, not dimple
 
 const dimple = await makeDimple({ config: "./dimple.jsonc" });
 
@@ -57,500 +39,60 @@ const mem = await dimple.write({
 });
 
 // hybrid semantic + FTS search
-const results = await dimple.query("how does the pipeline deploy", {
-  limit: 3,
-});
-
-// index real code — files + extracted functions/classes/methods become memories
-const indexed = await dimple.index({ paths: ["./src"] });
-
-await dimple.dispose(); // stop the worker, close DBs (idempotent)
-```
-
-## The problem & the approach
-
-**Problem**: naive memory stores scan every vector per query (cost grows
-with memory count), return flat results with no structure, and give the
-agent no notion of _why_ a memory is relevant, what supersedes it, or what
-contradicts it.
-
-**Approach** — three structures maintained automatically by a durable
-background worker:
-
-1. **Topic tree** — memories are routed by embedding similarity into tree
-   leaves. Queries descend the tree (beam search on centroids) and only
-   score the reached leaves: **search cost grows with depth, not with
-   memory count**. A durable worker splits overflowing leaves and merges
-   underflowing ones with deterministic algorithms (CF-triplet k-means) —
-   same embeddings ⇒ same tree, anywhere.
-2. **Hybrid retrieval** — vector (cosine) + FTS5 lexical legs fused with
-   Reciprocal Rank Fusion, optional recency boost, and per-result graph
-   context.
-3. **Artifact graph + concepts** — directed `memory_supports` /
-   `memory_supersedes` / `memory_contradicts` edges written by structural
-   rules, code indexing, the agent, and the optional **dream** job (LLM
-   classification of candidates + deterministic concept clustering).
-   Retrieval consumes the graph: superseded memories demote, conflicts
-   surface, supports ride along as provenance; concept filters give the
-   query sovereignty.
-
-All structural decisions are **deterministic**; everything is **durable**
-(two SQLite files, resume-safe jobs).
-
-## Configuration
-
-Pass a config file path or an inline object:
-
-```ts
-const dimple = await makeDimple({ config: "./dimple.jsonc" });
-// or — the inline object mirrors the JSON config exactly (quoted keys):
-const dimple = await makeDimple({
-  config: {
-    storeUrl: "file:./.dimple/dimple.db",
-    jobs: { dbUrl: "file:./.dimple/jobs.db" },
-    embedding: {
-      defaultModel: "minilm",
-      models: [
-        {
-          id: "minilm",
-          provider: "transformers",
-          model: "Xenova/all-MiniLM-L6-v2",
-          dimensions: 384,
-        },
-      ],
-    },
-  },
-});
-```
-
-### Config sections (`dimple.jsonc`)
+const hits = await dimple.query("how does the pipeline deploy", { limit: 3 });
 
-| Section                     | Fields                                                                                                                          | Notes                                      |
-| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ |
-| `storeUrl`                  | string                                                                                                                          | memory store DB (`file:` or remote libSQL) |
-| `jobs`                      | `dbUrl`, `execution` (reserved), `concurrency`, `leaseTimeoutMs` (reserved), `maxAttempts` (1–100), `pollIntervalMs` (reserved) | separate DB for the durable queue          |
-| `embedding`                 | `defaultModel` + `models[]`                                                                                                     | **required** — see Models below            |
-| `topics`                    | tree shape (`minLeafItems`, `maxLeafItems`, …)                                                                                  | optional, defaults apply                   |
-| `graph`                     | graph config                                                                                                                    | optional                                   |
-| `llm`                       | `baseURL`, `apiKey`, `modelId`, `temperature`                                                                                   | enables `dream`/`repair`                   |
-| `logging`                   | `enabled`, `level`, `format` (`jsonl`\|`pretty`), `file`, `scopes`, `ringSize`                                                  | silent by default; `file` = durable JSONL  |
-| `defaultK` / `defaultLimit` | numbers                                                                                                                         | retrieval defaults                         |
-| `defaultTtlSecs`            | number                                                                                                                          | write TTL default                          |
+// index a codebase (tree-sitter: one unit per file + per definition)
+await dimple.index({ paths: ["./src"] });
 
-### Remote databases (Turso)
-
-`storeUrl` and `jobs.dbUrl` accept **remote libSQL URLs** (`libsql://`,
-`https://`, `ws://` — Turso or any libsql-compatible host). The same
-engine runs everything the local file does — FTS5, vector search, the
-topic tree, durable jobs — over the wire. Verified end-to-end against
-the real Turso service (`tests/store/turso-remote.test.ts`).
-
-Turso tokens are **per-database**, so each URL carries its own token.
-`dimple.jsonc` resolves `${ENV_VAR}` in values, so tokens stay out of
-config files:
-
-```jsonc
-{
-  "storeUrl": "libsql://<org>-<memdb>.turso.io?authToken=${TURSO_MEMORIES_TOKEN}",
-  "jobs": {
-    "dbUrl": "libsql://<org>-<jobsdb>.turso.io?authToken=${TURSO_JOBS_TOKEN}",
-  },
-  "embedding": {
-    /* hosted providers for serverless — local models are
-                    local-only (auto-install needs Node) */
-  },
-}
-```
-
-Notes:
-
-- The **jobs DB must also be remote** when the store is remote (the
-  durable queue + workflow state are cross-invocation)
-- `file:`-only behaviors (parent-dir creation, migration lock) skip
-  for remote URLs
-- A single-DB setup can share one URL + one token (`TURSO_TOKEN`)
-- For Cloudflare Workers the client swaps to the pure-JS entry
-  (`@libsql/client/web` — one bundler alias, no source changes)
-
-### Embedding models
-
-Local embeddings (`provider: "transformers"`) auto-install on first
-use — `@huggingface/transformers` is an **optional peer** so hosted-only
-consumers never pull its tree (the `sharp` + `onnxruntime-node`
-postinstall scripts that abort npm 11 installs). The first `embed()`
-installs it with the project's own package manager, scripts skipped
-(proven unnecessary for embedding), and retries in-process. Offline or
-restricted environments get a typed `Sdk.Embedding` error with the manual
-command. Set `autoInstall: false` on the model to disable:
-`embedding.models[].autoInstall: false`
-
-### Embedding models
-
-Two providers — **local** (Transformers.js ONNX, runs on your CPU, no
-keys) and **hosted** (any OpenAI-compatible endpoint, plus native
-provider families):
-
-```jsonc
-"embedding": {
-  "defaultModel": "bge",
-  "models": [
-    { "id": "minilm", "provider": "transformers",
-      "model": "Xenova/all-MiniLM-L6-v2", "dimensions": 384 },
-    { "id": "bge", "provider": "transformers",
-      "model": "Xenova/bge-small-en-v1.5", "dimensions": 384 },
-    { "id": "bge-base", "provider": "transformers",
-      "model": "Xenova/bge-base-en-v1.5", "dimensions": 768 },
-    { "id": "gemini", "provider": "openai",
-      "model": "gemini-embedding-2", "dimensions": 3072, "apiKey": "…",
-      "baseURL": "https://generativelanguage.googleapis.com/v1beta/openai/" },
-    { "id": "openai", "provider": "openai",
-      "model": "text-embedding-3-small", "dimensions": 1536, "apiKey": "…" }
-  ]
-}
+await dimple.dispose(); // required before process.exit — the durable worker
 ```
 
-**Hosted providers** — the config `provider` field selects the family:
-
-| `provider`         | Models                                                                                                                                                  | `baseURL` needed?                                           |
-| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
-| `openai` (default) | any OpenAI-compatible endpoint — OpenAI, vLLM, **ollama** (`http://localhost:11434/v1`), Ollama Cloud, LM Studio, Together, Fireworks, Groq, Mistral, … | optional (defaults to OpenAI) — set it for custom endpoints |
-| `google`           | `gemini-embedding-001`, `gemini-embedding-2`, `gemini-embedding-2-preview` (native provider)                                                            | ❌ no                                                       |
-| `cohere`           | `embed-english-v3.0`, `embed-multilingual-v3.0`, … (native provider)                                                                                    | ❌ no                                                       |
-| `bedrock`          | `amazon.titan-embed-text-v2:0`, `amazon.nova-embed-text-v2:0`, `cohere.embed-*` (native provider; AWS creds from env)                                   | ❌ no                                                       |
-
-`dimensions` is sent as a provider option where the API supports it
-(openai-compatible/`outputDimensionality`/`outputDimension`/titan `dimensions`
-and nova `embeddingDimension`) and ALWAYS enforced on the output.
-Verified end-to-end against real `gemini-embedding-2` (3072-dim), local
-ollama `Qwen3-Embedding-0.6B` (1024-dim), and the openai-compatible
-contract.
-
-- `dimensions` must match the model's real output — a mismatch fails typed
-  on first use
-- Multiple models coexist: each gets its own vector tables
-  (`vec_<dims>_<model>`) and its own topic tree; queries route per model
-- **Per-model runtime options** (transformers provider, all optional):
-  `device` (`auto` | `gpu` | `cpu` | `wasm` | `webgpu` | `cuda` | `dml` |
-  `coreml` | `webnn` — `cpu`/`wasm` tested; GPU/CUDA needs the onnxruntime
-  CUDA libs, fails typed without them), `dtype` (`fp32` | `fp16` | `q8` |
-  `int8` | `uint8` | `q4` | `bnb4` | `q4f16` | `q2` | `q2f16` | `q1` |
-  `q1f16` — quantizes model WEIGHTS only, never the output vectors;
-  `fp32`/`q8`/`int8`/`q4`/`fp16` verified end-to-end, others depend on the
-  HF repo shipping that ONNX variant), `pooling` (`mean` | `cls` | `none`),
-  `normalize` (boolean), `cacheDir`
-- **DB-level vector quantization** via `vectorType` (`float32` |
-  `float16` | `float8` | `float1bit`): the per-memory search vectors are
-  stored as int8/bfloat16/binary blobs and searched with libSQL's native
-  `vector_distance_cos` — `float8` verified end-to-end at **395 B vs
-  1536 B (4× smaller)** for 384-dim. The topic tree's centroid math and
-  `memories.embedding` always stay float32.
-- **Switching models**: set `defaultModel` to a configured model (multi-
-  model configs coexist — each model gets its own tree + `vec_<dims>_<model>`
-  tables). After a restart the new default handles all writes/queries;
-  the old model's tree stays intact. Existing memories are backfilled via
-  the durable `memory.embed` job (re-embed + re-route into the new tree,
-  idempotent/resume-safe) — the store-level switch (`setActiveModel` +
-  `findStale`) drives it, reachable through the `{ core: … }` path. Until
-  backfilled, old memories remain in the old tree (queries descend the
-  ACTIVE model's tree), and the runtime is bound to one active model at a
-  time (per-model reads for another model fail typed with
-  `topics.ModelMismatch`).
-- Vectors are always stored as float32 (`F32_BLOB`, 4 B/dim: 384 = 1536 B,
-  768 = 3072 B) regardless of `dtype`
-
-### Power users: `{ core: … }`
-
-Instead of a config, pass full `CoreOptions` (store/jobs URLs, models,
-`defaultModelId`, `embedderFor` factory, topics/graph/logging/llm,
-`concurrency`, `maxAttempts`, `extraDefs`/`extraHandlers` for custom durable
-jobs — register `{ name: "my.job", handler }` and enqueue it via the
-runtime; `extraHandlers` entries override built-ins by name). This is the
-injection seam used by tests (synthetic embedders).
+Calling conventions: **plain arguments** (`query(text, opts)`,
+`write(input, opts)`, `index({ paths })`); every failure is a rejected
+promise with a closed `_tag` (`"Sdk.NotFound"`, `"Sdk.InputError"`, …);
+`dispose()` is idempotent.
 
 ## API surface
 
-### `write(input, options?)`
-
-```ts
-await dimple.write({
-  logicalKey: "deploy-1",        // required, non-empty
-  content: "…",                   // required
-  namespace?: "default",
-  contentType?: "text/plain",
-  ttlSecs?: 604800,
-  metadata?: "{\"json\":true}",   // JSON string
-  typedMetadata?: { … },
-}, { embed: true });              // embed + topic-route (default); false for externally-managed embeddings
-```
-
-Returns the persisted `MemoryUnit`: `{ id, logicalKey, content,
-contentType, state: "active" | "superseded" | …, createdAt, ttlSecs,
-namespace, metadata, embedding? }` (re-fetched after embedding). Duplicate
-active `logicalKey` → `Sdk.DuplicateKey` (carries `logicalKey`/`namespace`).
-
-**Memory expiry**: `ttlSecs` (or the config `defaultTtlSecs`) schedules
-expiry; a sweep job retires expired memories in the background.
+| Method                                                    | What it does                                                                                                                                                            |
+| --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `write(input, { embed })`                                 | Store a memory (idempotent per `logicalKey`)                                                                                                                            |
+| `writeMany(units, { embed })`                             | Bulk ingest: batch insert + batch embed + routing in one pass (skips the per-write candidate scan — quadratic under bulk; run the dream job afterwards for graph edges) |
+| `get(id)` / `forget(id)`                                  | Fetch / remove by id or logical key                                                                                                                                     |
+| `query(text, options)`                                    | Hybrid search — profiles, budgets, content-type filter                                                                                                                  |
+| `updateEmbedding(id, vec, modelKey?)`                     | Attach an external embedding                                                                                                                                            |
+| `index({ paths })`                                        | Codebase ingestion (idempotent re-index)                                                                                                                                |
+| `graph.addEdge/confirm/reject/expire/candidates/coverage` | Memory relationships: supports / supersedes / contradicts                                                                                                               |
+| `explain({ kind, id }, options)`                          | Bounded graph neighborhood                                                                                                                                              |
+| `dream(modelKey?)` / `repair(modelKey?)`                  | Consolidation + maintenance jobs                                                                                                                                        |
+| `health(modelKey?)`                                       | Topic-tree invariant health                                                                                                                                             |
+| `reload(input?)`                                          | Blue-green config reload                                                                                                                                                |
+
+Errors are a closed hierarchy (`InputError`, `NotFound`, `DuplicateKey`,
+`JobFailed`, `LLMNotConfigured`, `StoreFailure`) — see the docs site.
 
-`query(text, { budget: { maxUnits, maxChars, maxMs } })` — LLM-facing
-context guardrail: caps the row count, truncates result content to the
-total char budget, and marks `results.truncated` when the soft time cap
-is exceeded.
-
-### `get(id)` / `forget(id)`
-
-Fetch one memory; soft-delete (cascades edges, rebalances the tree).
-
-### `query(text, options?)`
-
-```ts
-await dimple.query("how does the pipeline deploy", {
-  limit?: 10,
-  k?: 60,                          // RRF smoothing
-  namespace?: "default",
-  profile?: {
-    recencyBoost?: 0,              // 0 = off; hyperbolically decays with age
-    modeWeights?: { vector?: 0.5, fts?: 0.5 },
-    conceptFilter?: string[],      // concept ids (query sovereignty)
-  } | "hybrid" | "semantic" | "keyword",
-  topics?: { topicKeys?: string[] },
-});
-```
-
-Returns `AugmentedMemory[]` — each entry carries the memory, the fused
-score, its 1-based `rank`, and the graph context (`supersededBy`,
-`conflicts`, `supportedBy`). Zero-result queries resolve to `[]` (the
-absent-vector-table case is an empty dense leg, never an error).
-
-### `index(options)` — codebase ingestion
-
-```ts
-await dimple.index({
-  paths: ["./src", "./packages"],  // dirs (recursive) and/or files
-  keyPrefix?: "code-",
-  namespace?: "default",
-  embed?: true,
-  maxFileBytes?: 64 * 1024,
-  maxChunkBytes?: 8 * 1024,
-  extensions?: [".ts", ".tsx", ".js", ".jsx", …],
-});
-// → { files, indexed, updated, reconciled, skipped, commit, unsupported, failed }
-```
-
-- Tree-sitter parse (TypeScript/JavaScript/TSX/JSX) → the extractor
-  produces one **file unit** per file (with its import graph in metadata)
-  and one **chunk unit** per definition (function/class/method/interface/
-  type/enum/variable, with kind/name/line-range/docstring metadata)
-- Every unit embeds + topic-routes (unless `embed: false`)
-- AST-derived graph: relative imports resolve to indexed files and
-  class→method containment become **confirmed `memory_supports` edges**
-  (`source: "rule"`)
-- **Idempotent + upserting**: keys are cwd-anchored + deterministic
-  (chunk keys drop the line number, so edits ABOVE a definition update
-  it instead of duplicating it). Re-indexing skips unchanged units
-  (`skipped`), and changed units are **superseded + rewritten under the
-  same key** (`updated`) with a confirmed identity `memory_supersedes`
-  edge — stale copies never accumulate
-- **Edges re-form after `forget`**: phase-2 resolves endpoints by logical
-  key against the store (not just the current run), so a re-created unit
-  re-connects to its pre-existing dependencies
-- **Reconcile + rename detection**: units whose source vanished (deleted
-  file/definition) are demoted (`reconciled`, `source_removed` — gone
-  from search; domain-scoped to the walked roots, so `index lib` never
-  touches other files). A vanished unit whose embedding is a near-duplicate
-  of a new unit becomes a PENDING `memory_supersedes` (reason rename) —
-  `graph confirm` completes it. Code units are TTL-exempt
-  (`ttlSecs: null` — the index owns their lifecycle); `commit` stamps the
-  repo HEAD into every unit's metadata as `codeVersion`
-- **`code_calls` edges**: every call site inside a definition resolves to
-  the callee's chunk (same file, then imported files) → confirmed edges —
-  `explain(id, { edgeKinds: ["code_calls"] })` gives one-hop impact
-  analysis
-- Per-file failures are collected, never fatal
-
-### `updateEmbedding(id, embedding, modelKey?)` / `embed(text)`
-
-Attach an externally-computed embedding (completes the `{ embed: false }`
-workflow — the memory becomes semantically retrievable + topic-routed), or
-embed text with the SDK's own pipeline.
-
-### `dream(modelKey)` / `repair(modelKey)`
-
-Run the consolidation job (concept clustering + LLM edge classification;
-requires the `llm` config section) or the graph repair pass. Each call
-gets a fresh run id (always executes). Failures are typed:
-`Sdk.LLMNotConfigured`, `Sdk.JobFailed`.
-
-### `explain({ kind: "memory" | "topic" | "concept", id }, options?)`
-
-Graph neighborhood traversal — `{ nodes: [{ kind, id, depth, edgeKind }] }`.
-`options.edgeKinds` scopes the answer (memory-only navigation: pass the
-`memory_*` kinds; `code_calls` shows the call graph).
-
-### `health(modelKey)` / `graph.*`
-
-- `health` — topic-tree invariant sweep (`warnCount` + violation strings)
-- `graph.confirm(id)` / `graph.reject(id)` / `graph.expire(id)` — the
-  pending-edge lifecycle: promote to confirmed (a `memory_supersedes`
-  confirm demotes the old endpoint in the same transaction), reject, or
-  soft-delete. `Sdk.EdgeNotFound` for stale ids
-- `graph.addEdge({ fromMemoryId, toMemoryId, edgeKind: "memory_supports" |
-"memory_supersedes" | "memory_contradicts", source: "rule" | "llm" |
-"agent", weight?, confidence?, confirmed? })` — explicit edges; typed
-  validation (no self-loops, known kinds, weight > 0, confidence ∈ [0,1])
-- `graph.candidates()` — pending structural candidates + isolated count
-- `graph.coverage(modelKey)` — coverage %, pending, concept count
-
-### `reload(input?)` — apply a config change
-
-Pauses and re-boots with the re-resolved config:
-
-1. the config source is re-resolved **typed** — a `dimple.jsonc` path is
-   re-read from disk, an inline object re-validated (a bad config rejects
-   the reload with **zero impact** — nothing pauses);
-2. an **identical** config resolves as `{ reloaded: false }` — no pause,
-   no restart (safe to call from a file watcher);
-3. new calls wait, in-flight calls settle on the old stack;
-4. a fresh stack boots (new worker fleet, embedders, logger — jobs are
-   durable, so an interrupted handler's lease expires and re-claims after
-   boot);
-5. the swap: the old stack is disposed, the new one serves.
-
-Resolves with `{ config, changed, reloaded }` — `changed` flags which
-sections actually moved (`storeUrl` / `jobsUrl` / `rest`).
-
-Works for `{ config: … }`-built dimples (reload with no argument re-reads
-the original source). `{ core: … }`-built dimples have no config source
-and reject with `Sdk.Reload`. Boot failure leaves the previous system
-running (blue-green); a drain that cannot settle in-flight calls (30s)
-rejects with `Sdk.Reload` and nothing is torn down.
-
-**What happens to data when the database URL changes:** the old database
-file is preserved on disk but no longer served — nothing is migrated or
-merged, the new db is bootstrapped empty (a warning is logged for the
-switch; roll back with `reload(oldConfig)`). Pending jobs in a switched
-`jobs.dbUrl` stay durable in the old db, re-claimable by switching back
-or draining it externally.
-
-```ts
-await dimple.reload();                        // re-read dimple.jsonc
-await dimple.reload({ storeUrl: "file:…", … }); // or a fresh config
-```
-
-### `dispose()`
-
-Stops the worker, closes sinks + the store. Idempotent; post-dispose calls
-reject with a closed `Sdk.StoreFailure`.
-
-## Errors — the closed hierarchy
-
-Every promise rejects with typed errors from one family (never raw Effect
-internals). Export surface: `ConfigError`, `NotFoundError`,
-`DuplicateKeyError` (with `logicalKey`/`namespace`), `DuplicateEdgeError`,
-`LLMNotConfiguredError`, `JobFailedError`, `StoreFailure`, `InputError`,
-plus the `remapError` helper (maps unknown failures onto the hierarchy).
-Also importable from the `@ponraaj/dimple-sdk/errors` subpath.
-
-**Type exports** (all importable from the package root): `Dimple` (the
-facade interface), `MakeDimpleOptions`, `ConfigInput`, `QueryOptions`,
-`IndexOptions`, `IndexResult`, plus the error classes and the `DimpleError`
-union type. Errors also carry stable `_tag` values (`Sdk.DuplicateKey`,
-`Sdk.NotFound`, `Sdk.InputError`, `Sdk.JobFailed`, `Sdk.LLMNotConfigured`,
-`Sdk.StoreFailure`, …) — the recommended discriminator.
-
-## Background jobs (what runs while you work)
-
-| Job                                               | Trigger             | Work                                                                                                                       |
-| ------------------------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------- |
-| `topic.split` / `topic.merge` / `topic.rebalance` | automatic per write | keep every leaf within its size budget (deterministic splits, donor borrow, nearest-sibling merge, uneven-split rebalance) |
-| `memory.embed`                                    | model backfill      | re-embed a memory with a target model + route into its tree                                                                |
-| `memory.dream`                                    | `dream()`           | concept clustering + LLM edge classification                                                                               |
-| `graph.repair`                                    | `repair()`          | graph invariant repair                                                                                                     |
-| `audit.write`                                     | query path          | async audit-log writes (never blocks queries) — one FUSED row per hybrid query with the per-leg breakdown                  |
-
-Jobs are durable (leased, retried up to `maxAttempts`, resume-safe across
-processes). Every `makeDimple` boots the worker; `dispose` stops it.
-
-Every `query` also writes a retrieval audit row (query, namespace, mode,
-per-leg counts/scores, sources) into `retrieval_log` — machine-readable
-query history, persisted asynchronously (never blocks the query).
-
-## Storage & logs
+## Configuration
 
-| What                           | Where                                                                                        |
-| ------------------------------ | -------------------------------------------------------------------------------------------- |
-| Memories, tree, graph, audit   | `storeUrl` (`file:…`)                                                                        |
-| Durable queue + workflow state | `jobs.dbUrl` (separate DB)                                                                   |
-| Logs                           | stdout + in-memory ring (4096) by default; durable JSONL **only** when `logging.file` is set |
-| Grammar WASMs                  | `~/.cache/dimple/tree-sitter/` (downloaded once, offline after)                              |
-| ONNX model                     | bun/npm cache (~90 MB, first use)                                                            |
+`makeDimple({ config: "./dimple.jsonc" })` — every section is optional
+except `embedding`. `dimple init` (CLI) writes the full annotated schema;
+`${ENV_VAR}` placeholders keep tokens out of files. The complete reference
+(store / jobs / embeddings / topics / graph / llm / logging / Turso) is on
+the docs site: [Configuration reference](https://dimple.ponraaj.me/reference/config).
 
-## Changelog
+## MCP server
 
-Every release creates a GitHub Release with the per-version changes (the
-tag-based publish workflow): repo → **Releases** page.
+`dimple mcp` (from the CLI package) exposes dimple to any MCP client over
+the 2026-07-28 protocol — API discovery plus sandboxed code execution with
+the whole API as a `dimple` namespace. Guide:
+[MCP server & code mode](https://dimple.ponraaj.me/guides/mcp).
 
 ## Agent integration patterns
 
-These are the shapes that work well when the consumer is another agent
-(a harness) rather than a human with a REPL.
-
-### Session-start injection
-
-The guaranteed-usage path: a warmup query returns a compact memory block
-the harness injects as context, and the pending-graph review list rides
-along:
-
-```ts
-const [context, pending] = await Promise.all([
-  dimple.query("ongoing work and decisions", { limit: 8, budget: { maxChars: 4000 } }),
-  dimple.graph.candidates(),
-]);
-// context  → inject the top-N content strings into the system prompt
-// pending  → offer each candidate as a confirm/reject decision
-```
-
-`graph.candidates()` pending `memory_supersedes` edges from the rename
-scanner (rule 0.9x near-duplicates) are the highest-value batch to review.
-
-### Supersedes recipe
+Session-start injection, the supersedes recipe, and namespaces as the
+code/message partition are covered on the docs site (Architecture section)
+and in `docs/MCP.md` in the repository.
 
-`addEdge({ … edgeKind: "memory_supersedes", confirmed: true })` performs
-the full state flip in one transaction (the old memory demotes
-immediately) — it is safe to use directly. For human-gated flows the
-pending → confirm pair keeps the old memory ranking until a reviewer
-agrees:
-
-```ts
-const edge = await dimple.graph.addEdge({
-  fromMemoryId: oldId,
-  toMemoryId: newId,
-  edgeKind: "memory_supersedes",
-  source: "agent",
-}); // pending — nothing changed yet
-await dimple.graph.confirm(edge.id); // ← the flip happens here
-```
-
-### Namespaces as the code/message partition
-
-Until a query-time content-type filter exists, namespaces give clean
-isolation: `index({ paths, namespace: "code" })` and
-`write({ …, namespace: "messages" })` partition the store; query with
-`namespace: "code"` for code and `namespace: "messages"` for
-conversations. (`query({ contentType })` now does the same within one
-namespace — the two levers compose.)
-
-### Remember
-
-- `dimple.reload(newConfig)` swaps the whole stack blue-green (validate →
-  pause → boot → drain → swap); a failed boot leaves the old stack
-  serving.
-- JSON output is compact by default: CLI `--json` omits embeddings (pass
-  `--embedding` to include them); the SDK always keeps them on
-  `memory.embedding`.
-
-## Notes
+## Changelog
 
-- Determinism: same data + same config ⇒ same tree, same clusters, same
-  audit hashes — reproducible across machines and runs
-- `migrationFolder` is accepted in `makeDimple` options for advanced users;
-  the default path uses migrations embedded at build time (nothing to ship)
+[Release history](https://dimple.ponraaj.me/changelog/sdk/) · repository
+[`CHANGELOG.md`](./CHANGELOG.md)

package/dist/dimple.d.ts

@@ -117,6 +117,12 @@ export interface Dimple {
      * `stats` use when no model is passed. */
     readonly defaultModelId: string;
     readonly write: (input: WriteMemoryInput, options?: WriteOptions) => Promise<MemoryUnit>;
+    /** Bulk write: batch insert + batch embed + topic routing in ONE pass.
+     * Skips the per-write graph candidate scan (O(leaf size) per write —
+     * quadratic under bulk ingest); callers that need near-duplicate/
+     * conflict edges on bulk content should run the dream job afterwards.
+     * All-or-nothing: a duplicate logical key rejects the whole batch. */
+    readonly writeMany: (units: readonly WriteMemoryInput[], options?: WriteOptions) => Promise<readonly MemoryUnit[]>;
     readonly get: (id: string) => Promise<MemoryUnit>;
     readonly query: (text: string, options?: QueryOptions) => Promise<AugmentedMemory[]>;
     readonly forget: (id: string) => Promise<void>;

package/dist/index.js

@@ -766,6 +766,7 @@ import { Effect as Effect7, Schema as Schema7 } from "effect";
 import { parse, parseTree, printParseErrorCode } from "jsonc-parser";
 var CONFIG_FILE_NAMES = ["dimple.jsonc", "dimple.json"];
 var GLOBAL_DIR = ".config/dimple";
+var resolveCwd = () => process.env["DIMPLE_CWD"] ?? process.cwd();
 var defaults = {
   storeUrl: "file:./.dimple/dimple.db",
   defaultK: 60,
@@ -871,7 +872,7 @@ var loadFs = () => Effect7.tryPromise(() => import("node:fs"));
 var findProjectConfig = () => Effect7.gen(function* () {
   const fs = yield* loadFs();
   const path = yield* Effect7.tryPromise(() => import("node:path"));
-  let dir = process.cwd();
+  let dir = resolveCwd();
   for (let i = 0;i < 20; i++) {
     for (const name of CONFIG_FILE_NAMES) {
       const p = path.join(dir, name);
@@ -940,6 +941,23 @@ var resolveEnvVars = (value) => {
   }
   return value;
 };
+var resolveCwdUrls = (value) => {
+  const cwd = process.env["DIMPLE_CWD"];
+  if (cwd === undefined)
+    return value;
+  if (typeof value === "string" && value.startsWith("file:./")) {
+    return `file:${cwd}${cwd.endsWith("/") ? "" : "/"}${value.slice("file:./".length)}`;
+  }
+  if (Array.isArray(value))
+    return value.map(resolveCwdUrls);
+  if (value !== null && typeof value === "object") {
+    const result = {};
+    for (const [k, v] of Object.entries(value))
+      result[k] = resolveCwdUrls(v);
+    return result;
+  }
+  return value;
+};
 var deepMerge = (base, over) => {
   const out = { ...base };
   for (const [key, value] of Object.entries(over)) {
@@ -957,7 +975,7 @@ var loadConfigFromFile = (path) => Effect7.gen(function* () {
   const parsed = yield* parseJsoncEffect(content, (cause) => new Error(`config parse ${path}: ${cause.message}`));
   const merged = deepMerge(defaults, parsed);
   const resolved = resolveEnvVars(merged);
-  return yield* Schema7.decodeUnknownEffect(DimpleConfig, { errors: "all" })(resolved).pipe(Effect7.mapError((e) => new Error(`config validation: ${e.message}`)));
+  return yield* Schema7.decodeUnknownEffect(DimpleConfig, { errors: "all" })(resolveCwdUrls(resolved)).pipe(Effect7.mapError((e) => new Error(`config validation: ${e.message}`)));
 });
 var loadConfig = (overrides) => Effect7.gen(function* () {
   const fileParts = yield* loadFromFiles();
@@ -970,7 +988,7 @@ var loadConfig = (overrides) => Effect7.gen(function* () {
   if (overrides)
     merged = deepMerge(merged, overrides);
   const resolved = resolveEnvVars(merged);
-  return yield* Schema7.decodeUnknownEffect(DimpleConfig, { errors: "all" })(resolved).pipe(Effect7.mapError((e) => new Error(`config validation: ${e.message}`)));
+  return yield* Schema7.decodeUnknownEffect(DimpleConfig, { errors: "all" })(resolveCwdUrls(resolved)).pipe(Effect7.mapError((e) => new Error(`config validation: ${e.message}`)));
 });
 
 // ../config/src/service.ts
@@ -1236,7 +1254,7 @@ var makeBackend = (url, client, db, migrationFolder) => ({
                ORDER BY f.rank
                LIMIT ?`,
       args: [
-        `"${query.replaceAll('"', '""')}"`,
+        query.split(/\s+/).filter((token) => token.length > 0).map((token) => `"${token.replaceAll('"', '""')}"`).join(" "),
         namespace,
         ...topicKeys && topicKeys.length > 0 ? [...topicKeys, modelKey ?? ""] : [],
         ...typeof contentType === "string" ? [contentType] : contentType ?? [],
@@ -4610,6 +4628,58 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
       embeddingModel: undefined
     };
   });
+  const writeMany = Effect26.fn("Store.writeMany")(function* (inputs) {
+    if (inputs.length === 0)
+      return [];
+    for (const input of inputs) {
+      if (input.content.includes("\x00")) {
+        return yield* new StoreError({
+          message: "Store.writeMany: content contains a NUL byte (libSQL would silently truncate it)"
+        });
+      }
+    }
+    const now = Date.now();
+    const rows = inputs.map((input) => {
+      const metadataBlob = input.typedMetadata !== undefined ? mergeMetadataBlob(input.metadata, input.typedMetadata) : input.metadata ?? null;
+      return {
+        id: crypto.randomUUID(),
+        logicalKey: input.logicalKey,
+        content: input.content,
+        contentType: input.contentType ?? "text/plain",
+        createdAt: now,
+        ttlSecs: input.ttlSecs === null ? null : input.ttlSecs ?? defaultTtlSecs,
+        namespace: input.namespace ?? "default",
+        metadata: metadataBlob
+      };
+    });
+    yield* Effect26.tryPromise({
+      try: () => db.transaction(async (tx) => {
+        await tx.insert(memories2).values(rows);
+        await tx.insert(memoryEvents2).values(rows.map((row) => ({
+          memoryId: row.id,
+          eventType: "created",
+          createdAt: now
+        })));
+      }),
+      catch: (cause) => backend.isUniqueViolation(cause) ? new DuplicateActiveKey({
+        logicalKey: inputs[0].logicalKey,
+        namespace: inputs[0].namespace ?? "default"
+      }) : new StoreError({ message: `Store.writeMany: ${cause}` })
+    });
+    return rows.map((row) => ({
+      id: row.id,
+      logicalKey: row.logicalKey,
+      content: row.content,
+      contentType: row.contentType,
+      state: "active",
+      createdAt: row.createdAt,
+      ttlSecs: row.ttlSecs,
+      namespace: row.namespace,
+      metadata: row.metadata ?? undefined,
+      embedding: undefined,
+      embeddingModel: undefined
+    }));
+  });
   const get = Effect26.fn("Store.get")(function* (id) {
     const rows = yield* Effect26.tryPromise({
       try: () => db.select(memoryFields).from(memories2).where(eq8(memories2.id, id)).limit(1),
@@ -4733,6 +4803,36 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
     if (!result)
       return yield* new NotFound({ id: key });
   });
+  const decrementTreeCf = (model, membership, x) => Effect26.tryPromise({
+    try: () => db.transaction(async (tx) => {
+      const leafRows = await tx.select({
+        topicKey: topics.topicKey,
+        memberCount: topics.memberCount,
+        sumSquares: topics.sumSquares
+      }).from(topics).where(and8(eq8(topics.topicKey, membership.topicKey), eq8(topics.modelKey, membership.modelKey)));
+      const leaf = leafRows[0];
+      if (!leaf)
+        return;
+      const next = cfRemove({
+        count: Number(leaf.memberCount),
+        ls: new Float32Array(0),
+        ss: Number(leaf.sumSquares)
+      }, x);
+      await tx.update(topics).set({
+        memberCount: Math.max(0, next.count),
+        sumSquares: Math.max(0, next.ss),
+        updatedAt: Date.now()
+      }).where(and8(eq8(topics.topicKey, membership.topicKey), eq8(topics.modelKey, membership.modelKey)));
+      const relRows = await tx.select({ parentKey: topicRelations.parentKey }).from(topicRelations).where(and8(eq8(topicRelations.childKey, membership.topicKey), eq8(topicRelations.modelKey, membership.modelKey), eq8(topicRelations.relationKind, "parent_child")));
+      let ancestorKey = relRows[0]?.parentKey;
+      while (ancestorKey) {
+        await tx.update(topics).set({ memberCount: sql7`max(0, ${topics.memberCount} - 1)`, updatedAt: Date.now() }).where(and8(eq8(topics.topicKey, ancestorKey), eq8(topics.modelKey, membership.modelKey)));
+        const up = await tx.select({ parentKey: topicRelations.parentKey }).from(topicRelations).where(and8(eq8(topicRelations.childKey, ancestorKey), eq8(topicRelations.modelKey, membership.modelKey), eq8(topicRelations.relationKind, "parent_child")));
+        ancestorKey = up[0]?.parentKey;
+      }
+    }),
+    catch: (cause) => new StoreError({ message: `Store: tree CF decrement: ${cause}` })
+  });
   const forget = Effect26.fn("Store.forget")(function* (id) {
     let membership;
     if (topicsEnabled) {
@@ -4746,37 +4846,7 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
         const vectorRows = yield* backend.fetchMemoryVectors([id], modelConfig);
         if (vectorRows.length > 0) {
           const x = new Float32Array(vectorRows[0].vector);
-          const now = Date.now();
-          yield* Effect26.tryPromise({
-            try: () => db.transaction(async (tx) => {
-              const leafRows = await tx.select({
-                topicKey: topics.topicKey,
-                memberCount: topics.memberCount,
-                sumSquares: topics.sumSquares
-              }).from(topics).where(and8(eq8(topics.topicKey, mem.topicKey), eq8(topics.modelKey, mem.modelKey)));
-              const leaf = leafRows[0];
-              if (!leaf)
-                return;
-              const next = cfRemove({
-                count: Number(leaf.memberCount),
-                ls: new Float32Array(0),
-                ss: Number(leaf.sumSquares)
-              }, x);
-              await tx.update(topics).set({
-                memberCount: Math.max(0, next.count),
-                sumSquares: Math.max(0, next.ss),
-                updatedAt: now
-              }).where(and8(eq8(topics.topicKey, mem.topicKey), eq8(topics.modelKey, mem.modelKey)));
-              const relRows = await tx.select({ parentKey: topicRelations.parentKey }).from(topicRelations).where(and8(eq8(topicRelations.childKey, mem.topicKey), eq8(topicRelations.modelKey, mem.modelKey), eq8(topicRelations.relationKind, "parent_child")));
-              let ancestorKey = relRows[0]?.parentKey;
-              while (ancestorKey) {
-                await tx.update(topics).set({ memberCount: sql7`max(0, ${topics.memberCount} - 1)`, updatedAt: now }).where(and8(eq8(topics.topicKey, ancestorKey), eq8(topics.modelKey, mem.modelKey)));
-                const up = await tx.select({ parentKey: topicRelations.parentKey }).from(topicRelations).where(and8(eq8(topicRelations.childKey, ancestorKey), eq8(topicRelations.modelKey, mem.modelKey), eq8(topicRelations.relationKind, "parent_child")));
-                ancestorKey = up[0]?.parentKey;
-              }
-            }),
-            catch: (cause) => new StoreError({ message: `Store.forget: tree CF: ${cause}` })
-          });
+          yield* decrementTreeCf(modelConfig, mem, x);
           yield* updateTopicVectorForget(backend, modelConfig, mem.topicKey, x);
         }
       }
@@ -4972,7 +5042,7 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
     }
     return yield* topicHealth(modelKey, topicsConfig).pipe(Effect26.provideService(Service2, backend));
   });
-  const updateEmbedding = Effect26.fn("Store.updateEmbedding")(function* (id, embedding, modelName) {
+  const updateEmbedding = Effect26.fn("Store.updateEmbedding")(function* (id, embedding, modelName, options) {
     const now = Date.now();
     const affected = yield* Effect26.tryPromise({
       try: () => db.transaction(async (tx) => {
@@ -5003,10 +5073,34 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
       return yield* new NotFound({ id });
     if (modelConfig && modelConfig.id === modelName) {
       yield* backend.ensureVectorTable(modelConfig);
+      if (topicsEnabled) {
+        const memberRows = yield* Effect26.tryPromise({
+          try: () => db.select({
+            topicKey: topicMemberships.topicKey,
+            modelKey: topicMemberships.modelKey
+          }).from(topicMemberships).where(eq8(topicMemberships.memoryId, id)),
+          catch: (cause) => new StoreError({ message: `Store.updateEmbedding: load membership: ${cause}` })
+        });
+        const mem = memberRows[0];
+        if (mem && mem.modelKey === modelName) {
+          const oldVec = yield* backend.fetchMemoryVectors([id], modelConfig);
+          if (oldVec.length > 0) {
+            const x = new Float32Array(oldVec[0].vector);
+            yield* decrementTreeCf(modelConfig, mem, x);
+            yield* updateTopicVectorForget(backend, modelConfig, mem.topicKey, x);
+          }
+          yield* Effect26.tryPromise({
+            try: () => db.delete(topicMemberships).where(and8(eq8(topicMemberships.memoryId, id), eq8(topicMemberships.modelKey, modelName))),
+            catch: (cause) => new StoreError({ message: `Store.updateEmbedding: delete membership: ${cause}` })
+          });
+        }
+      }
       yield* backend.insertVector(id, embedding, modelConfig);
       if (topicsEnabled) {
         const routed = yield* routeInsert(id, embedding, modelConfig, topicsConfig).pipe(Effect26.provideService(Service2, backend), Effect26.mapError((cause) => new StoreError({ message: `Store.updateEmbedding topic route: ${cause}` })));
-        yield* scanLeafCandidates(id, embedding, routed.topicKey, modelConfig);
+        if (options?.scanCandidates !== false) {
+          yield* scanLeafCandidates(id, embedding, routed.topicKey, modelConfig);
+        }
         if (routed.overflow) {
           const enqueueSplit = topicsOptions?.enqueueSplit;
           if (enqueueSplit) {
@@ -5566,6 +5660,7 @@ var makeStore2 = (backend, modelConfig, topicsOptions, graphConfig, auditOptions
   });
   return {
     write,
+    writeMany,
     writeSuperseding,
     reconcile,
     reconcileRenameCandidates,
@@ -6531,10 +6626,16 @@ import { existsSync as existsSync2, readFileSync } from "node:fs";
 import { join as join2 } from "node:path";
 import { pathToFileURL } from "node:url";
 
-// ../embedder/src/auto-install.ts
-import { execFile } from "node:child_process";
+// ../installer/src/index.ts
 import { existsSync } from "node:fs";
 import { join } from "node:path";
+
+class InstallerError extends Error {
+  constructor(message) {
+    super(message);
+    this.name = "InstallerError";
+  }
+}
 var detectPackageManager = (cwd) => {
   let dir = cwd;
   while (true) {
@@ -6549,38 +6650,74 @@ var detectPackageManager = (cwd) => {
       return { pm: "npm", root: dir };
     const parent = join(dir, "..");
     if (parent === dir)
-      return { pm: "bun", root: cwd };
+      return { pm: "npm", root: cwd };
     dir = parent;
   }
 };
-var installCommand = (pm) => {
-  switch (pm) {
-    case "bun":
-      return ["bun", "add", "@huggingface/transformers"];
-    case "npm":
-      return [
-        "npm",
-        "install",
-        "@huggingface/transformers",
-        "--ignore-scripts",
-        "--no-audit",
-        "--no-fund"
-      ];
-    case "pnpm":
-      return ["pnpm", "add", "@huggingface/transformers", "--ignore-scripts"];
-    case "yarn":
-      return ["yarn", "add", "@huggingface/transformers", "--ignore-scripts"];
-  }
+var resolveExecutable = (pm, platform = process.platform) => platform === "win32" && pm !== "bun" ? `${pm}.cmd` : pm;
+var installArgs = (pm, packages, root) => {
+  const argv = [pm === "npm" ? "install" : "add", ...packages];
+  if (pm !== "bun")
+    argv.push("--ignore-scripts");
+  if (pm === "npm")
+    argv.push("--no-audit", "--no-fund");
+  if (pm === "pnpm" && existsSync(join(root, "pnpm-workspace.yaml")))
+    argv.push("-w");
+  return argv;
 };
-var installTransformers = (cwd) => new Promise((resolve, reject) => {
+var installOnce = async (target, packages, opts) => {
+  const { spawn } = await import("node:child_process");
+  return new Promise((resolve, reject) => {
+    const timeoutMs = opts.timeoutMs ?? 300000;
+    const cmd = resolveExecutable(target.pm, opts.platform);
+    const args = installArgs(target.pm, packages, target.root);
+    const manual = `${cmd} ${args.join(" ")}`;
+    const child = spawn(cmd, args, { cwd: target.root, stdio: "ignore" });
+    let settled = false;
+    const timer = setTimeout(() => {
+      child.kill("SIGKILL");
+      if (settled)
+        return;
+      settled = true;
+      reject(new InstallerError(`auto-install timed out after ${timeoutMs} ms — the child was killed; run it manually: ${manual}`));
+    }, timeoutMs);
+    child.on("error", (cause) => {
+      clearTimeout(timer);
+      if (settled)
+        return;
+      settled = true;
+      reject(new InstallerError(`auto-install failed to start (${manual}): ${cause.message} — run it manually`));
+    });
+    child.on("exit", (code, signal) => {
+      clearTimeout(timer);
+      if (settled)
+        return;
+      settled = true;
+      if (code === 0) {
+        resolve();
+        return;
+      }
+      reject(new InstallerError(`auto-install failed (${manual} exited ${signal !== null ? `with ${signal}` : `code ${code ?? "?"}`}) — run it manually`));
+    });
+  });
+};
+var inflight = new Map;
+var installOptionalPeers = (packages, opts) => {
+  const cwd = opts?.cwd ?? process.cwd();
   const target = detectPackageManager(cwd);
-  const [cmd, ...args] = installCommand(target.pm);
-  execFile(cmd, args, { cwd: target.root, encoding: "utf8" }, (error) => {
-    if (error) {
-      reject(new Error(`auto-install of @huggingface/transformers failed (${target.pm} add …): ${error.message} — ` + "run it manually or set `autoInstall: false` and configure a hosted embedding provider"));
-      return;
-    }
-    resolve();
+  const key = `${target.root}\x00${packages.join(",")}`;
+  const existing = inflight.get(key);
+  if (existing !== undefined)
+    return existing;
+  const run = installOnce(target, packages, opts ?? {});
+  inflight.set(key, run);
+  run.then(() => inflight.delete(key), () => inflight.delete(key));
+  return run;
+};
+// ../embedder/src/auto-install.ts
+var installTransformers = (cwd) => installOptionalPeers(["@huggingface/transformers"], { cwd, timeoutMs: 300000 }).catch((cause) => {
+  throw new EmbeddingError({
+    message: cause instanceof InstallerError ? `${cause.message} — or set \`autoInstall: false\` and configure a hosted embedding provider` : `auto-install of @huggingface/transformers failed: ${String(cause)} — run it manually ` + "or set `autoInstall: false` and configure a hosted embedding provider"
   });
 });
 
@@ -6645,9 +6782,12 @@ var make5 = (options) => {
       }
       return Effect32.gen(function* () {
         yield* Effect32.logInfo("auto-installing @huggingface/transformers (first local-embed use — scripts skipped)").pipe(Effect32.annotateLogs({ scope: "@dimple/embedder", event: "embedder.auto-install" }));
-        yield* Effect32.tryPromise(() => installTransformers(process.cwd())).pipe(Effect32.timeout("5 minutes"), Effect32.catch((cause) => Effect32.fail(new EmbeddingError({
-          message: cause._tag === "TimeoutError" ? "auto-install of @huggingface/transformers timed out (5 minutes) — run " + "`bun add @huggingface/transformers` manually" : cause.cause instanceof Error ? cause.cause.message : cause instanceof Error ? cause.message : `auto-install failed: ${String(cause)}`
-        }))));
+        yield* Effect32.tryPromise(() => installTransformers(process.cwd())).pipe(Effect32.catch((cause) => {
+          const inner = cause.cause;
+          return Effect32.fail(inner instanceof EmbeddingError ? inner : new EmbeddingError({
+            message: `auto-install failed: ${inner instanceof Error ? inner.message : String(cause)}`
+          }));
+        }));
         return yield* Effect32.tryPromise(importFromDisk).pipe(Effect32.mapError(() => new EmbeddingError({
           message: "@huggingface/transformers still unavailable after auto-install — run `bun add " + "@huggingface/transformers` manually (or set `autoInstall: false` and configure a hosted embedding provider)"
         })));
@@ -6855,8 +6995,8 @@ var layer11 = (options) => Layer13.effect(Service10, Effect35.gen(function* () {
     const vWeight = profile.modeWeights?.vector ?? 1;
     const fWeight = profile.modeWeights?.fts ?? 1;
     const mode = vWeight === 0 && fWeight > 0 ? "keyword" : fWeight === 0 && vWeight > 0 ? "semantic" : "hybrid";
-    const denseRanks = vectorResults.length > 0 ? vectorResults.map((s) => [s.memory.id, s.score * vWeight]) : null;
-    const fts5Ranks = fts5Results.length > 0 ? fts5Results.map((s) => [s.memory.id, s.score * fWeight]) : null;
+    const denseRanks = vWeight > 0 && vectorResults.length > 0 ? vectorResults.map((s) => [s.memory.id, s.score * vWeight]) : null;
+    const fts5Ranks = fWeight > 0 && fts5Results.length > 0 ? fts5Results.map((s) => [s.memory.id, s.score * fWeight]) : null;
     if (!denseRanks && !fts5Ranks) {
       yield* store.logAudit({
         query: query2,
@@ -8395,8 +8535,8 @@ var walkFiles = (paths2, extensions) => {
   const files = [];
   const errors3 = [];
   let unsupported = 0;
-  const resolved = paths2.map((p) => resolve(p));
-  const root = commonRoot([...resolved, process.cwd()]);
+  const resolved = paths2.map((p) => resolve(process.env["DIMPLE_CWD"] ?? process.cwd(), p));
+  const root = commonRoot([...resolved, process.env["DIMPLE_CWD"] ?? process.cwd()]);
   const visit = (path2) => {
     let stat;
     try {
@@ -8789,6 +8929,30 @@ var makeFacade = (state, reload, dispose) => {
         return yield* invalid;
       return yield* writeUnit(input, options?.embed !== false);
     })),
+    writeMany: (units, options) => run(Effect49.gen(function* () {
+      if (!Array.isArray(units)) {
+        return yield* new InputError({ message: "writeMany: units must be an array" });
+      }
+      for (const input of units) {
+        const invalid = validateWriteInput(input);
+        if (invalid !== undefined)
+          return yield* invalid;
+      }
+      const mems = yield* core().store.writeMany(units);
+      if (options?.embed !== false) {
+        yield* core().store.ensureTopicTree(core().defaultModel.id);
+        const vectors = yield* core().defaultEmbedder.embed(mems.map((m) => m.content));
+        for (let i = 0;i < mems.length; i++) {
+          yield* core().store.updateEmbedding(mems[i].id, [...vectors[i]], core().defaultModel.id, {
+            scanCandidates: false
+          });
+        }
+        return yield* Effect49.forEach(mems, (mem) => core().store.get(mem.id), {
+          discard: false
+        });
+      }
+      return mems;
+    })),
     get: (id) => run(resolveMemory(id)),
     query: runQuery,
     forget: (id) => run(resolveMemory(id).pipe(Effect49.flatMap((memory) => core().store.forget(memory.id)))),
@@ -9089,7 +9253,7 @@ var makeDimple = async (options) => {
   let disposed = false;
   let gate = Promise.resolve();
   let releaseGate;
-  let inflight = 0;
+  let inflight2 = 0;
   let drainWaiters = [];
   let reloadChain = Promise.resolve();
   const run = (effect) => {
@@ -9101,12 +9265,12 @@ var makeDimple = async (options) => {
         return Promise.reject(new StoreFailure({ message: "SDK disposed" }));
       }
       const stack = current;
-      inflight += 1;
+      inflight2 += 1;
       return stack.runPromise(effect.pipe(Effect50.mapError(remapError))).catch((cause) => {
         return Promise.reject(remapError(cause));
       }).finally(() => {
-        inflight -= 1;
-        if (inflight === 0 && drainWaiters.length > 0) {
+        inflight2 -= 1;
+        if (inflight2 === 0 && drainWaiters.length > 0) {
           const waiters = drainWaiters;
           drainWaiters = [];
           for (const wake of waiters)
@@ -9127,7 +9291,7 @@ var makeDimple = async (options) => {
     release?.();
   };
   const drain = () => {
-    if (inflight === 0)
+    if (inflight2 === 0)
       return Promise.resolve();
     return new Promise((resolve2) => {
       drainWaiters.push(resolve2);

package/dist/internal/embedder/auto-install.d.ts

@@ -1,35 +1,10 @@
-/**
- * First-use auto-install of the OPTIONAL `@huggingface/transformers` peer.
- *
- * `@huggingface/transformers` is an optional peer (its tree pulls the
- * `sharp` + `onnxruntime-node` postinstall scripts that abort npm 11
- * installs). When the config declares a `transformers` provider model,
- * the intent is explicit — so the FIRST local-embed use installs the
- * package automatically with the project's own package manager, instead
- * of failing with a manual step.
- *
- * Safety notes:
- * - The install is scoped to the detected project root (the manifest's
- *   directory) and uses the manager the project already uses.
- * - EVERY install runs `--ignore-scripts` (npm/pnpm/yarn) or bun's
- *   default script-blocking — verified harmless: the onnxruntime-node
- *   binaries ship in the package; its postinstall is not required for
- *   embedding, and sharp is never loaded for text embeddings.
- * - Failure (offline, restricted env) is a typed EmbeddingError with the
- *   manual command — never a crash.
- */
-export type PackageManager = "bun" | "npm" | "pnpm" | "yarn";
-export interface InstallTarget {
-    readonly pm: PackageManager;
-    readonly root: string;
-}
-/** Detect the project's package manager by walking UP from `cwd` for a
- * lockfile (bun.lock/bun.lockb → bun, package-lock.json → npm,
- * pnpm-lock.yaml → pnpm, yarn.lock → yarn). Falls back to bun + cwd. */
-export declare const detectPackageManager: (cwd: string) => InstallTarget;
-/** The install argv for a manager. `--ignore-scripts` on npm/pnpm/yarn:
- * the blocked postinstalls are proven unnecessary for embedding (bun
- * blocks them by default already). */
+import { type PackageManager } from "./internal/installer/index.js";
+export type { InstallTarget, PackageManager } from "./internal/installer/index.js";
+export { detectPackageManager } from "./internal/installer/index.js";
+/** The install argv for a manager with the transformers package (kept for
+ * callers/tests that assert the shape). */
 export declare const installCommand: (pm: PackageManager) => readonly [string, ...string[]];
-/** Run the install for the project at `cwd`; rejects on non-zero exit. */
+/** Install transformers at the project root detected from `cwd`. The
+ * shared installer kills the child on timeout and dedups concurrent
+ * first-use installs; failures reject with the manual command. */
 export declare const installTransformers: (cwd: string) => Promise<void>;

package/dist/internal/installer/index.d.ts

@@ -0,0 +1,26 @@
+export type PackageManager = "bun" | "npm" | "pnpm" | "yarn";
+export interface InstallTarget {
+    readonly pm: PackageManager;
+    readonly root: string;
+}
+export declare class InstallerError extends Error {
+    constructor(message: string);
+}
+/** Detect the project's package manager by walking UP from `cwd` for a
+ * lockfile. NO lockfile → npm (the most universal — a bun fallback broke
+ * Node-only machines with ENOENT). */
+export declare const detectPackageManager: (cwd: string) => InstallTarget;
+/** The executable name for a manager. win32 ships `.cmd` shims for
+ * npm/pnpm/yarn — the bare names fail with ENOENT. */
+export declare const resolveExecutable: (pm: PackageManager, platform?: NodeJS.Platform) => string;
+/** The base install argv for a manager. Scripts are skipped everywhere;
+ * pnpm at a workspace root additionally gets `-w`. */
+export declare const installArgs: (pm: PackageManager, packages: readonly string[], root: string) => string[];
+export interface InstallOptions {
+    readonly cwd?: string;
+    /** Hard deadline; the child is SIGKILLed when it trips (default 5 min). */
+    readonly timeoutMs?: number;
+    /** Injectable for tests (defaults to process.platform). */
+    readonly platform?: NodeJS.Platform;
+}
+export declare const installOptionalPeers: (packages: readonly string[], opts?: InstallOptions) => Promise<void>;

package/dist/internal/store/service.d.ts

@@ -65,6 +65,11 @@ export interface ExplainResult {
 }
 export interface Interface {
     readonly write: (input: WriteMemoryInput) => Effect.Effect<MemoryUnit, StoreError | DuplicateActiveKey>;
+    /** Bulk write: batch insert in ONE transaction (all-or-nothing; a
+     * duplicate logical key fails the whole batch with DuplicateActiveKey).
+     * The per-write graph candidate scan is NOT part of this — pair with
+     * `updateEmbedding(..., { scanCandidates: false })` for bulk ingest. */
+    readonly writeMany: (inputs: readonly WriteMemoryInput[]) => Effect.Effect<readonly MemoryUnit[], StoreError | DuplicateActiveKey>;
     /** Upsert for the index path — the identity rule (same logical key ⇒
      * confirmed `memory_supersedes`) made reachable for ACTIVE keys: demote
      * the active holder of the key (if any), write the new unit, and link
@@ -139,7 +144,9 @@ export interface Interface {
     readonly forget: (id: string) => Effect.Effect<void, NotFound | StoreError>;
     readonly sweep: (namespace: string) => Effect.Effect<SweepResult, StoreError>;
     readonly events: (memoryId: string) => Effect.Effect<readonly MemoryEvent[], StoreError>;
-    readonly updateEmbedding: (id: string, embedding: ReadonlyArray<number>, model: string) => Effect.Effect<void, NotFound | StoreError>;
+    readonly updateEmbedding: (id: string, embedding: ReadonlyArray<number>, model: string, options?: {
+        readonly scanCandidates?: boolean;
+    }) => Effect.Effect<void, NotFound | StoreError>;
     readonly findStale: (model: string) => Effect.Effect<readonly string[], StoreError>;
     readonly createConcept: (name: string, embedding: ReadonlyArray<number>) => Effect.Effect<Concept, StoreError>;
     readonly getConcept: (id: string) => Effect.Effect<Concept, NotFound | StoreError>;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ponraaj/dimple-sdk",
-  "version": "0.0.0-alpha.6",
+  "version": "0.0.0-alpha.8",
   "private": false,
   "files": [
     "dist"