@theokit/sdk-cache 0.3.1 → 1.0.0

package/CHANGELOG.md

@@ -1,5 +1,169 @@
 # Changelog
 
+## 1.0.0
+
+### Major Changes
+
+- 643192a: **Breaking:** `CacheEmbedderError` is removed.
+
+  Nothing ever constructed it, so `catch (e) { if (e instanceof CacheEmbedderError) … }` was a branch
+  that could not run. Every embedder failure — on `consult`, on `remember`, and on both plugin hooks —
+  degrades to a cache miss or a skipped write, warns on stderr, and increments
+  `CacheStats.embedderFailures`, because a cache is an optimisation and must not take the request
+  down with it. That counter is how a broken embedder is detected, and the README now says so.
+
+  Nothing to migrate: any handler for this class was already dead code.
+
+### Minor Changes
+
+- ad8f9b9: The `"json"` persistence backend now keeps the promise it was sold on.
+
+  `Cache.ready()` — which the code's own comment referred to long before it existed — resolves once
+  the snapshot has been read, and `consult` / `remember` await hydration themselves, so a lookup
+  issued right after construction no longer races the read and misses an entry that is on disk.
+
+  `Cache.flush()` writes the debounced snapshot and keeps every entry. Writes are debounced 200ms, so
+  a once-per-invocation CLI — the process this backend exists for — used to persist nothing unless it
+  happened to live longer, and `clear()` was the only public call that forced a write. Nothing
+  flushes on teardown: call `flush()` before exiting.
+
+  Two caches built with the same `dir` and `namespace` now share one store instead of each writing a
+  full snapshot and erasing the other's entries. The first construction's `maxEntries` applies.
+
+### Patch Changes
+
+- e3f2a82: Public-API documentation reviewed file by file, and corrected wherever it disagreed
+  with the code. The docblocks ship in the `.d.ts`, so these read as behaviour changes
+  in an editor even though no behaviour changed.
+
+  The corrections that change what a caller would do:
+
+  - **`sdk-cache` documented its own premise backwards.** The header example labelled a
+    semantic hit as if it avoided the provider call. `asPlugin()` returns the cached
+    answer as `recalledContext`, which the agent loop injects as a `<memory-context>`
+    block _before_ the prompt — the request still goes to the provider. The two modes
+    are now labelled separately, with a table saying which one short-circuits and which
+    one seeds.
+  - **`sdk-handoff`'s five error classes said "throw".** Under the plugin wiring the
+    handler never throws; every failure becomes a tool result `{"ok":false,…}` handed
+    back to the model. Each class now says where it is actually observable. The header
+    also told readers to `import { Handoff } from "@theokit/sdk"`, from which it was
+    extracted.
+  - **`sdk-budget`'s `charge()` claimed idempotency across concurrent calls.** The mutex
+    serialises, it does not deduplicate: two identical calls record twice. Related, and
+    newly documented: with `maxUsd` set, a model missing from the pricing table denies
+    every request rather than passing it — and the table matches by exact string, so
+    `"openai/gpt-4o"` does not match `"gpt-4o"`.
+  - **The three `memory-*` adapters advertised an env-var fallback they do not read**,
+    and their peer dependencies are required rather than optional. Their behavioural
+    differences are now stated where they break the "interchangeable adapter"
+    assumption — honcho ignores `k` and always throws on `delete`; mem0 recalls across
+    sessions by design; supermemory ignores `sessionId` entirely.
+  - **`sdk-memory`'s `truncated` flag was documented as its own inverse**, and its
+    dreaming sweep claimed a mutex it never takes against the writer it names.
+  - **`sdk-tools`** corrected `run_vitest`'s unreachable `no_vitest` code, `truncation`'s
+    replacement-character claim, and two return shapes missing a live error code.
+  - **`acp`/`cli`** corrected sixteen statements including a named error class that is
+    not the one raised, a handler documented as calling `fork()` that refuses
+    unconditionally, handlers described as pure that mint ids and mutate a store, a
+    config loader credited to Zod in a package that does not import it, and a `--force`
+    scaffold described as atomic that deletes the destination before the rename.
+
+  Undocumented public symbols were documented across every package, with each claim
+  checked against the implementation rather than inferred from the name.
+
+- e368fc1: Every published declaration file now compiles without `skipLibCheck` (#345). The
+  DTS rollup emitted symbols as a re-export from a chunk while omitting them from
+  that chunk's `import`, and dropped type-only imports from external packages —
+  leaving 51 unresolved references across ten of the twelve packages. Nothing broke
+  at runtime, and `tsc` stayed green for anyone with `skipLibCheck` on, but a
+  consumer running type-aware lint saw every type reached through one degrade to
+  `error`.
+
+  The declarations are repaired at build time from the compiler's own diagnostics.
+  No source or API change.
+
+- e699569: **The repository moved to the official `usetheokit` organization.** Every `repository`, `bugs` and `homepage` field now points there, along with the README, `CONTRIBUTING.md`, `SECURITY.md` and the issue templates. Existing clones and any URL already published keep working — GitHub redirects a transferred repository permanently — so this is a correctness fix for the metadata npm renders, not a break.
+
+  **The Apache-2.0 text every package ships was replaced with the official one.** The copy distributed until now had paragraph 4(d) truncated: it read "except as required for describing the origin of the Work and reproducing the content of the NOTICE file", dropping "reasonable and customary use" from the licensed clause. §4(d) governs what a redistributor must do with attribution notices, and the omission narrowed it.
+
+  That matters more than a typo would. The manifests declare the SPDX identifier `Apache-2.0`, which is an assertion that the terms are _the_ Apache-2.0 terms — a licence scanner resolves the identifier and never reads the file. A consumer's compliance review, which does read the file, would find a body that no longer matches the identifier and has no name of its own. Every `LICENSE` in this repository is now byte-identical to the canonical text, with the appendix filled in.
+
+  Nothing else about the terms changed: the licence is the same licence it has always been meant to be, and no package changes what it grants.
+
+- e3f2a82: `@opentelemetry/api` is now declared as an optional peer dependency, so the spans these two
+  packages emit can actually reach a collector.
+
+  Both lazily `require("@opentelemetry/api")` from their own directory, but neither manifest
+  declared it in any dependency field. Under an isolated `node_modules` layout the specifier is
+  therefore not linked under the package, the require throws, the loader caches a `null` tracer,
+  and every span degrades to a no-op — silently, with no warning, unlike `@theokit/sdk`, which
+  prints one when telemetry is enabled and OTel is absent. For `sdk-cache` that covered both of
+  its main paths (`cache.lookup` on every send, `cache.store` on every reply), so an operator
+  reading a trace saw no cache activity at all and had no way to tell that from a cache that was
+  never consulted.
+
+  The declaration matches `@theokit/sdk`'s: `peerDependencies` plus `peerDependenciesMeta.optional`,
+  so nothing is installed for anyone who does not want OTel, and users who do want it get their
+  copy linked where the require can find it.
+
+- 8d1feaa: `PostAssistantReplyContext` now carries `usedTools`, and `@theokit/sdk-cache` stops caching
+  tool-using turns in plugin mode.
+
+  The cache's D266/EC-10 guard exists because replaying an answer produced by a `write_file` / HTTP
+  POST / payment call re-serves the text without the side effect having happened. The
+  `post_assistant_reply` hook had no tool signal to key on and passed a literal `false`, so the guard
+  never fired on the path that runs automatically — only a hand-written `cache.remember(..., {
+usedTools: true })` reached it.
+
+  The runtime derives the flag from the run's replayed event stream. A hook handler written against
+  the previous shape keeps working; code that CONSTRUCTS a `PostAssistantReplyContext` (test doubles,
+  custom emitters) now has to supply the field.
+
+- c7385d2: Test runs no longer claim every core on the host.
+
+  None of the package configs capped `maxWorkers`, so vitest's default applied: `os.availableParallelism()`,
+  one fork per core, each booting a full test environment. The repo's `test` script is
+  `turbo run test --filter='./packages/*'`, so that default is paid once per package _concurrently_ —
+  nproc forks times turbo's concurrency, on nproc cores. Measured on a 12-thread machine during an
+  unrelated investigation, two vitest pools alone were enough to reach load average 33.89 with the
+  desktop unusable; a full fan-out is several times that.
+
+  `@theokit/sdk` is the interesting case. B-104 recorded on 2026-08-19 that the `poolOptions.forks.*`
+  block was 100% dead in Vitest 4, deleted it, and noted that `fileParallelism: false` was forcing
+  `maxWorkers` to 1 unconditionally, so a fork-count knob could not act. B-059 then flipped
+  `fileParallelism` to `true` on 2026-08-20, which made the knob able to act again — and nothing
+  reintroduced one, so the package silently went back to the uncapped default. That comment has been
+  corrected along with the config; it claimed no knob existed, which is no longer true.
+
+  The cap leaves 4 cores free (`Math.max(2, cpus().length - 4)`), scaling with the runner rather than
+  hard-coding one machine's core count. It costs no wall-clock: measured in `theokit-ui`, the full
+  suite ran 73.96s at 4 workers against 74.36s at 12, so the parallelism above the cap was already
+  noise. Verified as resolved config rather than as file contents — `createVitest` reports
+  `maxWorkers: 8` on a 12-thread host, which is the formula, not the default.
+
+  This changes no published behaviour; it is test tooling only. Refs usetheokit/theokit-ui#51.
+
+## 0.3.2
+
+### Patch Changes
+
+- 8790f70: Refuse a `workspace:` range before it can reach npm.
+
+  Five of this repo's twelve publishable packages declare internal dependencies as `workspace:^`, which
+  is correct on disk and becomes an unrecoverable defect if the publish goes out through a tool that
+  does not rewrite it: `pnpm` resolves the protocol while packing, `npm` ships the manifest verbatim.
+  A version published that way fails to install for everyone and cannot be corrected — only
+  deprecated.
+
+  Every publishable package now runs the guard in `prepublishOnly`, so it fires whichever way the
+  publish is invoked, and `pnpm release` runs it once across the repo before `changeset publish`.
+
+  Note for anyone reading a published manifest: the `prepublishOnly` entry points at a path inside
+  this repository. It never runs for a consumer — the hook only fires when the package itself is
+  published — and guarding the entry point that a hand-run `npm publish` actually uses was worth the
+  cosmetic wart of shipping the line.
+
 ## 0.3.1
 
 ### Patch Changes

package/LICENSE

@@ -137,8 +137,8 @@
 
    6. Trademarks. This License does not grant permission to use the trade
       names, trademarks, service marks, or product names of the Licensor,
-      except as required for describing the origin of the Work and
-      reproducing the content of the NOTICE file.
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
 
    7. Disclaimer of Warranty. Unless required by applicable law or
       agreed to in writing, Licensor provides the Work (and each

package/README.md

@@ -72,7 +72,7 @@ Clears the in-memory state. For the JSON file store, deletes the persisted state
 
 ## Errors
 
-- `CacheEmbedderError` — embedder runtime failed (rate-limited, network error, etc.). Agent run continues without cache.
+An embedder failure never surfaces as an error: every path degrades to a cache miss or a skipped write, warns on stderr, and increments `CacheStats.embedderFailures`. That counter is how you detect a broken embedder.
 - `CacheInvalidTtlError` — TTL string failed parse (e.g., `"-5m"` or `"abc"`). Thrown pre-construction.
 
 ## How it fits with `@theokit/sdk`
@@ -96,7 +96,20 @@ import { Agent } from "@theokit/sdk";
 import { Cache } from "@theokit/sdk-cache";
 ```
 
-See `docs/migration/1-x-to-2-0.md` in the monorepo root.
+See the monorepo `CHANGELOG.md` for the 1.x → 2.0 package-split migration notes.
+
+## API reference
+
+Every symbol this package exports, with the exact specifier to import it from, is in the generated
+capability map that ships inside `@theokit/sdk`:
+
+```
+node_modules/@theokit/sdk/docs/harness-capability-map.md   # symbol -> import specifier
+node_modules/@theokit/sdk/docs/error-codes.md              # every `code` an error can carry
+```
+
+Both are generated from the built type declarations, so they describe the version you installed
+rather than the version someone wrote a page about.
 
 ## License
 

package/dist/index.cjs

@@ -1,12 +1,12 @@
 'use strict';
 
+var path = require('path');
 var sdk = require('@theokit/sdk');
 var persistence = require('@theokit/sdk/persistence');
 var zod = require('zod');
 var crypto = require('crypto');
 var module$1 = require('module');
 var promises = require('fs/promises');
-var path = require('path');
 
 var _documentCurrentScript = typeof document !== 'undefined' ? document.currentScript : null;
 // src/cache.ts
@@ -277,14 +277,6 @@ var InMemoryCacheStore = class {
 };
 
 // src/types/cache.ts
-var CacheEmbedderError = class extends Error {
-  name = "CacheEmbedderError";
-  cause;
-  constructor(message, cause) {
-    super(`Cache embedder failed: ${message}`);
-    if (cause !== void 0) this.cause = cause;
-  }
-};
 var CacheInvalidTtlError = class extends Error {
   constructor(input) {
     super(
@@ -524,13 +516,14 @@ var DEFAULT_TTL = { default: "1h" };
 var DEFAULT_NAMESPACE = "global";
 var DEFAULT_MAX_ENTRIES = 1e3;
 var Cache = class _Cache {
-  constructor(embedder, threshold, ttl, namespace, modelId, store) {
+  constructor(embedder, threshold, ttl, namespace, modelId, store, hydrated) {
     this.embedder = embedder;
     this.threshold = threshold;
     this.ttl = ttl;
     this.namespace = namespace;
     this.modelId = modelId;
     this.store = store;
+    this.hydrated = hydrated;
   }
   embedder;
   threshold;
@@ -538,7 +531,54 @@ var Cache = class _Cache {
   namespace;
   modelId;
   store;
+  hydrated;
   _plugin;
+  /**
+   * Resolves once the `"json"` backend has finished reading its snapshot; resolves immediately on
+   * the in-memory backend.
+   *
+   * You rarely need it: `consult` and `remember` await hydration themselves, so a cache is correct
+   * without it. It exists for a caller who wants the read charged to startup rather than to the
+   * first lookup — and because the code promised it long before it existed (#359).
+   *
+   * A corrupt or unreadable snapshot resolves normally with an empty cache and a warning on
+   * stderr; a cache must not take the process down.
+   */
+  async ready() {
+    await this.hydrated;
+  }
+  /**
+   * Write the pending snapshot to disk now, keeping every entry. No-op on the in-memory backend.
+   *
+   * Writes are debounced 200ms, so a process that remembers something and exits inside that window
+   * persists nothing — precisely the once-per-invocation CLI the `"json"` backend exists for. Call
+   * this before exiting. Nothing flushes on teardown: an `exit` handler cannot await, and a library
+   * installing a process-level hook is a side effect the caller did not ask for.
+   *
+   * Until #359 the only public call that wrote the snapshot was `clear()`, which also destroyed
+   * everything you wanted to persist.
+   */
+  async flush() {
+    const store = this.store;
+    if (typeof store.flush === "function") await store.flush();
+  }
+  /**
+   * Build a cache. Validates `options` with Zod and THROWS `ZodError` on a bad shape — an
+   * `embedder` missing `{ id, dimension, embed }`, a `threshold` outside `0..2`, a `namespace`
+   * longer than 64 chars, or `persistence: { backend: "json" }` without a `dir`.
+   *
+   * Defaults: `threshold` 0.85, `ttl` `{ default: "1h" }`, `namespace` `"global"`, `maxEntries`
+   * 1000 (LRU), `persistence` in-memory. `modelId` defaults to the literal string `"unknown"`,
+   * which is a real namespace value and not a wildcard: entries stored while `modelId` was
+   * defaulted are only ever returned to lookups that also default it.
+   *
+   * With `persistence: { backend: "json", dir }` the snapshot is read in the background and this
+   * call does not await it — but `consult` and `remember` do, so a lookup issued immediately after
+   * construction still sees what is on disk. Await {@link Cache.ready} to charge the read to
+   * startup instead of to the first lookup. Two caches built with the same `dir` and `namespace`
+   * share one store, so they cannot overwrite each other's snapshot; the FIRST one's `maxEntries`
+   * is the one that applies.
+   */
   static semantic(options) {
     CacheSemanticOptionsSchema.parse(options);
     const threshold = options.threshold ?? DEFAULT_THRESHOLD;
@@ -546,12 +586,28 @@ var Cache = class _Cache {
     const namespace = options.namespace ?? DEFAULT_NAMESPACE;
     const modelId = options.modelId ?? "unknown";
     const maxEntries = options.maxEntries ?? DEFAULT_MAX_ENTRIES;
-    const store = createStore(namespace, maxEntries, options.persistence);
-    return new _Cache(options.embedder, threshold, ttl, namespace, modelId, store);
+    const { store, hydrated } = createStore(namespace, maxEntries, options.persistence);
+    return new _Cache(options.embedder, threshold, ttl, namespace, modelId, store, hydrated);
   }
   /**
-   * EC-4 absorbed: memoized so repeated `asPlugin()` calls return the SAME
-   * plugin descriptor — no duplicate hook registration.
+   * A `Plugin` for `Agent.create({ plugins: [...] })` that reads the cache before each user turn
+   * and writes it after each assistant reply.
+   *
+   * READ THIS BEFORE BUDGETING FOR IT. A hit does NOT skip the model call. The hook returns the
+   * cached response as `PreUserSendResult.recalledContext`, which the agent loop injects as a
+   * `<memory-context>` block ahead of the prompt — the request still goes to the provider, still
+   * costs tokens, and still returns whatever the model makes of that context, which need not be
+   * the cached text. Use {@link Cache.consult} / {@link Cache.remember} when the point is to avoid
+   * the call.
+   *
+   * A turn that invoked tools is NOT cached: the store hook reads
+   * `PostAssistantReplyContext.usedTools`, which the runtime derives from the run's tool calls
+   * (#358). Replaying such an answer would hand a later caller the result of a write that never
+   * happened. Until that signal existed the hook passed a literal `false` and cached those turns,
+   * despite the package's stated intent.
+   *
+   * Memoized: repeated calls return the SAME plugin, so registering it twice does not double the
+   * hooks.
    */
   asPlugin() {
     if (this._plugin !== void 0) return this._plugin;
@@ -586,7 +642,13 @@ var Cache = class _Cache {
           await performStore({
             prompt: c.prompt,
             response: c.reply,
-            usedTools: false,
+            // EC-10 / D266 — never cache a reply the run produced with tools; replaying it hands a
+            // later caller the RESULT of a write without the write having happened. This used to
+            // be the literal `false` under a comment describing a `tool_call_id` heuristic that was
+            // never implemented, so the guard fired only for a hand-written `remember(...)` call
+            // and never on the plugin path, which is the one that runs automatically. #358 added
+            // the signal to `PostAssistantReplyContext`; this reads it.
+            usedTools: c.usedTools,
             store: cache.store,
             embedder: cache.embedder,
             ttl: cache.ttl,
@@ -600,13 +662,28 @@ var Cache = class _Cache {
     return this._plugin;
   }
   /**
-   * Explicit cache lookup — callers that want true LLM short-circuit
-   * call this BEFORE `agent.send()`, then dispatch to the LLM only on miss.
+   * Look a prompt up. Call it BEFORE dispatching to the model and skip the call on a hit — this is
+   * the only path in this package that actually avoids an LLM request.
+   *
+   * ```ts
+   * const hit = await cache.consult(prompt);
+   * if (hit.hit) return hit.response;
+   * ```
+   *
+   * `source` says which stage matched: `"kv"` is an exact-key match and costs NO embedding call;
+   * `"semantic"` means the prompt was embedded and a stored vector came within `threshold`, and
+   * only then is `distance` present (cosine distance, so smaller is closer).
    *
-   * v1 plugin mode provides recall + context-inject (LLM still called).
-   * v1.x will add transparent short-circuit via an agent-loop refactor.
+   * NEVER THROWS on an embedder failure. It degrades to `{ hit: false }`, logs a warning on
+   * stderr, and increments {@link CacheStats.embedderFailures} — a cache must not take the request
+   * down with it. A cache that has silently stopped hitting is that counter climbing, not a cold
+   * cache.
+   *
+   * An empty or whitespace-only prompt returns `{ hit: false }` and counts as a MISS. A prompt
+   * matching {@link CacheTTLConfig.exclude} returns `{ hit: false }` and counts as `excluded`.
    */
   async consult(prompt) {
+    await this.hydrated;
     const result = await performLookup({
       prompt,
       store: this.store,
@@ -627,10 +704,25 @@ var Cache = class _Cache {
     return { hit: false };
   }
   /**
-   * Explicit cache store — pair with `consult()` to manually feed the
-   * cache after dispatching the LLM call yourself.
+   * Store a prompt/response pair. Pair it with {@link Cache.consult} after you dispatched the model
+   * call yourself.
+   *
+   * Pass `{ usedTools: true }` when the answer came from a run that invoked tools and replaying it
+   * would lose the side effects — the write is then skipped entirely.
+   *
+   * Silently writes nothing when the prompt is empty/whitespace, the response is empty, the prompt
+   * matches {@link CacheTTLConfig.exclude}, or the embedder fails (that last case increments
+   * {@link CacheStats.embedderFailures}). It resolves in every one of those cases: a resolved
+   * promise is not evidence that an entry exists — read {@link CacheStats.entries} if you need
+   * that.
+   *
+   * Writing beyond `maxEntries` evicts the least-recently-used entry. On the `"json"` backend the
+   * disk write is DEBOUNCED by 200 ms, so a process that exits right after this resolves loses the
+   * entry unless it calls {@link Cache.flush} — which writes the snapshot and keeps every entry.
+   * Nothing flushes on teardown.
    */
   async remember(prompt, response, opts) {
+    await this.hydrated;
     await performStore({
       prompt,
       response,
@@ -642,27 +734,52 @@ var Cache = class _Cache {
       modelId: this.modelId
     });
   }
-  /** Stats snapshot — primary observable for dogfood verification. */
+  /**
+   * Counter snapshot for this instance. See {@link CacheStats} for what each counter separates —
+   * in particular `misses` vs `excluded` vs `embedderFailures`, which is how you tell a cold cache
+   * from a too-broad exclude regex from a broken embedder.
+   *
+   * Process-local: the `"json"` backend persists entries, never counters, so a restart reports
+   * zeros against a warm file.
+   */
   stats() {
     return this.store.stats();
   }
-  /** Clear all entries (and flush to disk if JSON backend). */
+  /**
+   * Drop every entry. On the `"json"` backend this also forces the debounced snapshot to disk
+   * immediately, so it is the one public call that guarantees the file matches memory.
+   *
+   * Counters are NOT reset — `stats()` keeps reporting the hits and misses accumulated before the
+   * clear, so `entries: 0` alongside a non-zero `kvHits` is expected, not a bug.
+   */
   async clear() {
     await this.store.clear();
   }
-  /** Force-evict expired entries. Returns count removed. */
+  /**
+   * Remove every entry whose TTL has elapsed, returning how many were dropped, and add that to
+   * {@link CacheStats.evicted}.
+   *
+   * Optional housekeeping: expired entries are already skipped on lookup and dropped when touched,
+   * so this only reclaims memory for entries nobody asks for. `now` exists to make the sweep
+   * testable; leave it out in production.
+   */
   evictExpired(now = Date.now()) {
     return this.store.evictExpired(now);
   }
 };
+var jsonStores = /* @__PURE__ */ new Map();
 function createStore(namespace, maxEntries, persistence) {
   if (persistence?.backend === "json") {
     const dir = persistence.dir;
+    const key = `${path.resolve(dir)}\0${namespace}`;
+    const existing = jsonStores.get(key);
+    if (existing !== void 0) return existing;
     const store = new JsonFileCacheStore(dir, namespace, maxEntries);
-    void store.hydrate();
-    return store;
+    const entry = { store, hydrated: store.hydrate() };
+    jsonStores.set(key, entry);
+    return entry;
   }
-  return new InMemoryCacheStore(maxEntries);
+  return { store: new InMemoryCacheStore(maxEntries), hydrated: Promise.resolve() };
 }
 
 // src/lexical-embedder.ts
@@ -688,7 +805,6 @@ function createLexicalEmbedder(dimension = 256) {
 }
 
 exports.Cache = Cache;
-exports.CacheEmbedderError = CacheEmbedderError;
 exports.CacheInvalidTtlError = CacheInvalidTtlError;
 exports.createLexicalEmbedder = createLexicalEmbedder;
 //# sourceMappingURL=index.cjs.map