@betterdb/semantic-cache 0.2.0 → 0.5.0

package/README.md

@@ -96,6 +96,54 @@ A lookup is a **hit** when `score <= threshold`. The default threshold is `0.1`.
 | Conversational / RAG | `0.15` | Paraphrases hit as `high` confidence |
 | Broad search / recall | `0.20` | High hit rate, review uncertain hits |
 
+## LLM-as-judge
+
+When a hit lands in the uncertainty band (`threshold - uncertaintyBand < score <= threshold`), you can supply a `judgeFn` to adjudicate automatically instead of handling `confidence: 'uncertain'` yourself.
+
+```typescript
+const result = await cache.check(userPrompt, {
+  judge: {
+    judgeFn: async ({ prompt, response, similarity, threshold, category }) => {
+      // Return true to accept (confidence → 'high')
+      // Return false to reject (treated as miss with nearestMiss)
+      const verdict = await openai.chat.completions.create({
+        model: 'gpt-5-mini',
+        messages: [
+          { role: 'system', content: 'Reply YES or NO only.' },
+          { role: 'user', content: `Does this cached response correctly answer the prompt?\nPrompt: ${prompt}\nResponse: ${response}` },
+        ],
+      });
+      return verdict.choices[0].message.content?.startsWith('YES') ?? false;
+    },
+    onError: 'accept',  // fail-open on judge errors (default)
+    timeoutMs: 2000,    // per-call timeout (default)
+  },
+});
+```
+
+**When the judge is invoked:** only for `confidence === 'uncertain'` hits. High-confidence hits, misses, and the zero-candidates case bypass the judge entirely.
+
+**Accept path:** `result.hit === true`, `result.confidence === 'high'`.
+
+**Reject path:** `result.hit === false`, `result.nearestMiss` populated with `deltaToThreshold <= 0` (use this to distinguish judge rejections from regular misses where `deltaToThreshold > 0`).
+
+**Composing with rerank:** when both `rerank` and `judge` are set, the judge receives the reranked pick's response and similarity score.
+
+**`checkBatch()` does not support `judge`.** Call `check()` individually for prompts that need adjudication.
+
+### CacheCheckOptions reference
+
+| Option | Type | Default | Description |
+|---|---|---|---|
+| `threshold` | `number` | `defaultThreshold` | Per-request cosine distance threshold override |
+| `category` | `string` | — | Category tag for per-category thresholds and metric labels |
+| `filter` | `string` | — | FT.SEARCH pre-filter expression (trusted input only) |
+| `k` | `number` | `1` | KNN neighbours to fetch (ignored when `rerank` is set) |
+| `staleAfterModelChange` | `boolean` | `false` | Evict and miss when stored model differs from `currentModel` |
+| `currentModel` | `string` | — | Model to compare against stored entries |
+| `rerank` | `RerankOptions` | — | Rerank hook; see `RerankOptions` |
+| `judge` | `JudgeOptions` | — | LLM-as-judge for borderline hits; see `JudgeOptions`. Not supported by `checkBatch()`; throws `SemanticCacheUsageError` |
+
 ## Configuration Reference
 
 | Option | Type | Default | Description |
@@ -161,6 +209,24 @@ Cost savings scale with the model. Observed values from live examples:
 | `@betterdb/semantic-cache/embed/cohere` | `embed-english-v3.0` | 1024 |
 | `@betterdb/semantic-cache/embed/ollama` | `nomic-embed-text` | 768 |
 
+### Discovery markers
+
+Starting in `0.2.0`, `initialize()` writes a small advisory record to a shared `__betterdb:caches` hash on the Valkey instance so Monitor (and other tooling) can enumerate caches without configuration. A 60s-TTL heartbeat key is refreshed every 30s; `flush()` and `dispose()` remove the heartbeat immediately. No sensitive data is ever written — only cache metadata (type, prefix, version, capabilities, configured thresholds).
+
+Opt out by passing `discovery: { enabled: false }`. See `SemanticCacheOptions.discovery` for the full set of knobs.
+
+If your Valkey runs with ACLs, grant the library's user access to the `__betterdb:*` prefix:
+
+```
+ACL SETUSER <user> +@write +@read ~__betterdb:* ~<your-cache-prefix>:*
+```
+
+Discovery writes are best-effort — if the ACL denies them, the cache still functions and the `semantic_cache_discovery_write_failed_total` counter increments so operators can alert.
+
+### `cache.dispose()`
+
+Graceful shutdown: stops the heartbeat and deletes this instance's heartbeat key so Monitor marks the cache offline immediately. Does not drop the index or delete entries. Call from your SIGTERM handler alongside `client.quit()`.
+
 ## API
 
 ### `cache.initialize()`
@@ -215,7 +281,15 @@ Returns `{ name, numDocs, dimension, indexingState }`.
 
 ### `cache.flush()`
 
-Drops the index and all keys. Call `initialize()` again to rebuild.
+Drops the index and all entries. Call `initialize()` again to rebuild. Also stops the discovery heartbeat and deletes its heartbeat key, but preserves the registry entry in `__betterdb:caches` so Monitor retains history.
+
+### `cache.shutdown()`
+
+Stops the analytics client, cancels the stats snapshot timer, and disposes the discovery heartbeat. Safe to call multiple times.
+
+### `cache.dispose()`
+
+Graceful shutdown of the discovery layer for in-process caches without destroying data. Stops the discovery heartbeat and deletes the heartbeat key; does not touch the index or entries.
 
 ### `cache.thresholdEffectiveness(options?)`
 

package/dist/SemanticCache.d.ts

@@ -8,15 +8,22 @@ export declare class SemanticCache {
     private readonly entryPrefix;
     private readonly statsKey;
     private readonly similarityWindowKey;
-    private readonly defaultThreshold;
+    private readonly configKey;
+    private defaultThreshold;
     private readonly defaultTtl;
-    private readonly categoryThresholds;
+    private categoryThresholds;
     private readonly uncertaintyBand;
     private readonly telemetry;
     private readonly costTable;
     private readonly embeddingCacheEnabled;
     private readonly embeddingCacheTtl;
     private readonly embedKeyPrefix;
+    private readonly discoveryOptions;
+    private readonly _initialDefaultThreshold;
+    private readonly _initialCategoryThresholds;
+    private readonly configRefreshOptions;
+    private configRefreshTimer;
+    private discovery;
     private _initialized;
     private _dimension;
     private _hasBinaryRefs;
@@ -40,8 +47,18 @@ export declare class SemanticCache {
     constructor(options: SemanticCacheOptions);
     initialize(): Promise<void>;
     flush(): Promise<void>;
-    /** Shut down the analytics client and cancel the stats timer. */
+    /**
+     * Shut down the analytics client, cancel the stats timer, and stop the
+     * discovery heartbeat. Safe to call multiple times.
+     */
     shutdown(): Promise<void>;
+    /**
+     * Graceful shutdown of the discovery layer — stops the heartbeat and
+     * deletes this instance's heartbeat key so Monitor marks the cache offline
+     * immediately. Does NOT touch the registry hash, the FT index, or any
+     * entries. Safe to call multiple times.
+     */
+    dispose(): Promise<void>;
     check(prompt: string | ContentBlock[], options?: CacheCheckOptions): Promise<CacheCheckResult>;
     store(prompt: string | ContentBlock[], response: string, options?: CacheStoreOptions): Promise<string>;
     /**
@@ -82,8 +99,29 @@ export declare class SemanticCache {
     thresholdEffectivenessAll(options?: {
         minSamples?: number;
     }): Promise<ThresholdEffectivenessResult[]>;
+    /**
+     * Refresh threshold config from Valkey. Returns true on a successful HGETALL,
+     * false if the call threw.
+     *
+     * Field semantics:
+     *   - "threshold"            -> updates defaultThreshold
+     *   - "threshold:{category}" -> updates categoryThresholds[category]
+     *   - "threshold:" (empty)   -> ignored
+     *   - non-numeric values     -> ignored
+     *   - out-of-range values    -> ignored (must be 0 <= x <= 2)
+     *
+     * Categories present in memory but absent from the hash fall back to their
+     * constructor values (or are removed if no constructor override existed).
+     * The default threshold likewise falls back to its constructor value if
+     * `threshold` is absent from the hash.
+     */
+    refreshConfig(): Promise<boolean>;
     /** @internal Default similarity threshold. */
     get _defaultThreshold(): number;
+    /** @internal Test-only getter. */
+    get _categoryThresholds(): Readonly<Record<string, number>>;
+    /** @internal Test-only getter. */
+    get _configRefreshIntervalMs(): number;
     /**
      * Execute a stable FT.SEARCH for use by adapters (e.g. LangGraph).
      * SORTBY inserted_at ASC gives stable ordering across paginated calls.
@@ -98,7 +136,9 @@ export declare class SemanticCache {
         vector: number[];
         durationSec: number;
     }>;
+    private startConfigRefresh;
     private _doInitialize;
+    private registerDiscovery;
     private initAnalyticsSafe;
     private captureStatsSnapshot;
     private ensureIndexAndGetDimension;

package/dist/SemanticCache.js

@@ -10,7 +10,9 @@ const utils_1 = require("./utils");
 const defaultCostTable_1 = require("./defaultCostTable");
 const cluster_1 = require("./cluster");
 const analytics_1 = require("./analytics");
+const discovery_1 = require("./discovery");
 const INVALIDATE_BATCH_SIZE = 1000;
+const PACKAGE_VERSION = require('../package.json').version;
 function errMsg(err) {
     return err instanceof Error ? err.message : String(err);
 }
@@ -22,6 +24,7 @@ class SemanticCache {
     entryPrefix;
     statsKey;
     similarityWindowKey;
+    configKey;
     defaultThreshold;
     defaultTtl;
     categoryThresholds;
@@ -31,6 +34,12 @@ class SemanticCache {
     embeddingCacheEnabled;
     embeddingCacheTtl;
     embedKeyPrefix;
+    discoveryOptions;
+    _initialDefaultThreshold;
+    _initialCategoryThresholds;
+    configRefreshOptions;
+    configRefreshTimer;
+    discovery = null;
     _initialized = false;
     _dimension = 0;
     _hasBinaryRefs = false;
@@ -59,6 +68,7 @@ class SemanticCache {
         this.entryPrefix = `${this.name}:entry:`;
         this.statsKey = `${this.name}:__stats`;
         this.similarityWindowKey = `${this.name}:__similarity_window`;
+        this.configKey = `${this.name}:__config`;
         this.embedKeyPrefix = `${this.name}:embed:`;
         this.defaultThreshold = options.defaultThreshold ?? 0.1;
         this.defaultTtl = options.defaultTtl;
@@ -85,6 +95,16 @@ class SemanticCache {
         });
         this.analyticsOpts = options.analytics;
         this.usesDefaultCostTable = useDefault;
+        this.discoveryOptions = options.discovery ?? {};
+        // Capture constructor values as fallback when __config fields are absent
+        this._initialDefaultThreshold = this.defaultThreshold;
+        this._initialCategoryThresholds = { ...this.categoryThresholds };
+        // Refresh options
+        const refresh = options.configRefresh ?? {};
+        this.configRefreshOptions = {
+            enabled: refresh.enabled ?? true,
+            intervalMs: Math.max(1000, refresh.intervalMs ?? 30_000),
+        };
     }
     // -- Lifecycle --
     async initialize() {
@@ -102,6 +122,14 @@ class SemanticCache {
         this._initialized = false;
         this._initPromise = null;
         this._initGeneration++;
+        // Capture and null the discovery ref synchronously, before any await,
+        // so a concurrent _doInitialize() (started after _initGeneration++) can't
+        // race in and have its new manager overwritten by this flush.
+        const discoveryToStop = this.discovery;
+        this.discovery = null;
+        if (discoveryToStop) {
+            await discoveryToStop.stop({ deleteHeartbeat: true });
+        }
         // Valkey Search 1.2 does not support the DD (Delete Documents) flag on
         // FT.DROPINDEX. Drop the index first, then clean up keys separately.
         try {
@@ -126,14 +154,41 @@ class SemanticCache {
         await this.client.del(this.similarityWindowKey);
         this.analytics.capture('cache_flush');
     }
-    /** Shut down the analytics client and cancel the stats timer. */
+    /**
+     * Shut down the analytics client, cancel the stats timer, and stop the
+     * discovery heartbeat. Safe to call multiple times.
+     */
     async shutdown() {
         this.shutdownCalled = true;
+        if (this.configRefreshTimer) {
+            clearInterval(this.configRefreshTimer);
+            this.configRefreshTimer = undefined;
+        }
         if (this.statsTimer) {
             clearInterval(this.statsTimer);
             this.statsTimer = undefined;
         }
         await this.analytics.shutdown();
+        await this.dispose();
+    }
+    /**
+     * Graceful shutdown of the discovery layer — stops the heartbeat and
+     * deletes this instance's heartbeat key so Monitor marks the cache offline
+     * immediately. Does NOT touch the registry hash, the FT index, or any
+     * entries. Safe to call multiple times.
+     */
+    async dispose() {
+        if (this.configRefreshTimer) {
+            clearInterval(this.configRefreshTimer);
+            this.configRefreshTimer = undefined;
+        }
+        if (this._initPromise) {
+            await this._initPromise.catch(() => { });
+        }
+        if (this.discovery) {
+            await this.discovery.stop({ deleteHeartbeat: true });
+            this.discovery = null;
+        }
     }
     // -- Public operations --
     async check(prompt, options) {
@@ -259,14 +314,85 @@ class SemanticCache {
                     return { hit: false, confidence: 'miss' };
                 }
             }
-            // All checks passed — record as a genuine hit
+            // All checks passed — compute confidence (recordSimilarityWindow moves to after judge)
+            let confidence = winnerScore >= threshold - this.uncertaintyBand ? 'uncertain' : 'high';
+            const matchedKey = winner.key;
+            // --- LLM-as-judge for borderline hits ---
+            if (options?.judge && confidence === 'uncertain') {
+                const judgeStart = performance.now();
+                const timeoutMs = options.judge.timeoutMs ?? 2000;
+                const onError = options.judge.onError ?? 'accept';
+                let decision;
+                try {
+                    const accepted = await raceWithTimeout(options.judge.judgeFn({
+                        prompt: promptText,
+                        response: winner.fields['response'] ?? '',
+                        similarity: winnerScore,
+                        threshold,
+                        category: category || undefined,
+                    }), timeoutMs);
+                    decision = accepted ? 'accept' : 'reject';
+                }
+                catch (err) {
+                    const isTimeout = err instanceof JudgeTimeoutError;
+                    if (onError === 'accept') {
+                        decision = isTimeout ? 'timeout_accept' : 'error_accept';
+                    }
+                    else {
+                        decision = isTimeout ? 'timeout_reject' : 'error_reject';
+                    }
+                }
+                const judgeSec = (performance.now() - judgeStart) / 1000;
+                this.telemetry.metrics.judgeDecisions
+                    .labels({ cache_name: this.name, category: categoryLabel, decision })
+                    .inc();
+                this.telemetry.metrics.judgeDuration
+                    .labels({ cache_name: this.name, category: categoryLabel, decision })
+                    .observe(judgeSec);
+                span.setAttributes({
+                    'cache.judge.invoked': true,
+                    'cache.judge.decision': decision,
+                    'cache.judge.latency_ms': judgeSec * 1000,
+                });
+                if (decision === 'accept') {
+                    confidence = 'high';
+                    // Fall through to hit-return path
+                }
+                else if (decision === 'error_accept' || decision === 'timeout_accept') {
+                    // Preserve 'uncertain'; fall through to hit-return path
+                }
+                else {
+                    // reject / error_reject / timeout_reject → treat as miss
+                    await this.recordSimilarityWindow(winnerScore, 'miss', category);
+                    await this.recordStat('misses');
+                    this.telemetry.metrics.requestsTotal
+                        .labels({ cache_name: this.name, result: 'miss', category: categoryLabel })
+                        .inc();
+                    span.setAttributes({
+                        'cache.hit': false,
+                        'cache.name': this.name,
+                        'cache.category': categoryLabel,
+                    });
+                    return {
+                        hit: false,
+                        confidence: 'miss',
+                        similarity: winnerScore,
+                        nearestMiss: {
+                            similarity: winnerScore,
+                            threshold,
+                            deltaToThreshold: winnerScore - threshold,
+                            matchedKey,
+                        },
+                    };
+                }
+            }
+            // --- End judge ---
+            // Record as genuine hit (moved here from before the judge block)
             await this.recordSimilarityWindow(winnerScore, 'hit', category);
-            const confidence = winnerScore >= threshold - this.uncertaintyBand ? 'uncertain' : 'high';
             await this.recordStat('hits');
             const metricResult = confidence === 'uncertain' ? 'uncertain_hit' : 'hit';
             this.telemetry.metrics.requestsTotal
                 .labels({ cache_name: this.name, result: metricResult, category: categoryLabel }).inc();
-            const matchedKey = winner.key;
             if (this.defaultTtl !== undefined && matchedKey) {
                 await this.client.expire(matchedKey, this.defaultTtl);
             }
@@ -446,6 +572,9 @@ class SemanticCache {
         if (options?.staleAfterModelChange) {
             throw new errors_1.SemanticCacheUsageError("checkBatch() does not support 'staleAfterModelChange'. Use check() for stale-model eviction.");
         }
+        if (options?.judge) {
+            throw new errors_1.SemanticCacheUsageError("checkBatch() does not support the 'judge' option. Use check() for LLM-as-judge adjudication.");
+        }
         return this.traced('checkBatch', async (span) => {
             // Resolve all prompts and embed in parallel
             const resolved = await Promise.all(prompts.map((p) => this.resolvePrompt(p)));
@@ -769,9 +898,64 @@ class SemanticCache {
         ]);
         return results;
     }
+    /**
+     * Refresh threshold config from Valkey. Returns true on a successful HGETALL,
+     * false if the call threw.
+     *
+     * Field semantics:
+     *   - "threshold"            -> updates defaultThreshold
+     *   - "threshold:{category}" -> updates categoryThresholds[category]
+     *   - "threshold:" (empty)   -> ignored
+     *   - non-numeric values     -> ignored
+     *   - out-of-range values    -> ignored (must be 0 <= x <= 2)
+     *
+     * Categories present in memory but absent from the hash fall back to their
+     * constructor values (or are removed if no constructor override existed).
+     * The default threshold likewise falls back to its constructor value if
+     * `threshold` is absent from the hash.
+     */
+    async refreshConfig() {
+        let raw = null;
+        try {
+            raw = await this.client.hgetall(this.configKey);
+        }
+        catch {
+            return false;
+        }
+        let nextDefault = this._initialDefaultThreshold;
+        const nextCategory = { ...this._initialCategoryThresholds };
+        if (raw) {
+            for (const [field, value] of Object.entries(raw)) {
+                const parsed = Number(value);
+                if (!Number.isFinite(parsed) || parsed < 0 || parsed > 2) {
+                    continue;
+                }
+                if (field === 'threshold') {
+                    nextDefault = parsed;
+                }
+                else if (field.startsWith('threshold:')) {
+                    const category = field.slice('threshold:'.length);
+                    if (category.length > 0) {
+                        nextCategory[category] = parsed;
+                    }
+                }
+            }
+        }
+        this.defaultThreshold = nextDefault;
+        this.categoryThresholds = nextCategory;
+        return true;
+    }
     // -- Internal helpers exposed to package adapters --
     /** @internal Default similarity threshold. */
     get _defaultThreshold() { return this.defaultThreshold; }
+    /** @internal Test-only getter. */
+    get _categoryThresholds() {
+        return this.categoryThresholds;
+    }
+    /** @internal Test-only getter. */
+    get _configRefreshIntervalMs() {
+        return this.configRefreshOptions.intervalMs;
+    }
     /**
      * Execute a stable FT.SEARCH for use by adapters (e.g. LangGraph).
      * SORTBY inserted_at ASC gives stable ordering across paginated calls.
@@ -788,19 +972,86 @@ class SemanticCache {
         return this.embed(text);
     }
     // -- Private helpers --
+    startConfigRefresh() {
+        if (!this.configRefreshOptions.enabled) {
+            return;
+        }
+        const tick = () => {
+            this.refreshConfig()
+                .then((ok) => {
+                if (!ok) {
+                    this.telemetry.metrics.configRefreshFailed
+                        .labels({ cache_name: this.name })
+                        .inc();
+                }
+            })
+                .catch(() => {
+                this.telemetry.metrics.configRefreshFailed
+                    .labels({ cache_name: this.name })
+                    .inc();
+            });
+        };
+        // Synchronous first refresh: process started immediately after a proposal
+        // was applied picks up the change without waiting for the first tick.
+        tick();
+        this.configRefreshTimer = setInterval(tick, this.configRefreshOptions.intervalMs);
+        if (typeof this.configRefreshTimer.unref === 'function') {
+            this.configRefreshTimer.unref();
+        }
+    }
     async _doInitialize() {
         const gen = this._initGeneration;
         return this.traced('initialize', async () => {
             const { dim, hasBinaryRefs } = await this.ensureIndexAndGetDimension();
-            if (this._initGeneration !== gen)
+            if (this._initGeneration !== gen) {
                 return;
+            }
             this._dimension = dim;
             this._hasBinaryRefs = hasBinaryRefs;
+            // registerDiscovery() may throw SemanticCacheUsageError on a name
+            // collision. Mark the cache initialized only after discovery succeeds
+            // so a colliding caller cannot subsequently call check()/store()
+            // against another owner's keys.
+            const manager = await this.registerDiscovery();
+            if (this._initGeneration !== gen) {
+                if (manager) {
+                    await manager.stop({ deleteHeartbeat: true });
+                }
+                return;
+            }
+            this.discovery = manager;
             this._initialized = true;
+            this.startConfigRefresh();
             // Fire analytics init once (not on every flush+initialize cycle)
             this.initAnalyticsSafe().catch(() => { });
         });
     }
+    async registerDiscovery() {
+        if (this.discoveryOptions.enabled === false) {
+            return null;
+        }
+        const metadata = (0, discovery_1.buildSemanticMetadata)({
+            name: this.name,
+            version: PACKAGE_VERSION,
+            defaultThreshold: this.defaultThreshold,
+            categoryThresholds: this.categoryThresholds,
+            uncertaintyBand: this.uncertaintyBand,
+            includeCategories: this.discoveryOptions.includeCategories ?? true,
+        });
+        const manager = new discovery_1.DiscoveryManager({
+            client: this.client,
+            name: this.name,
+            metadata,
+            heartbeatIntervalMs: this.discoveryOptions.heartbeatIntervalMs,
+            onWriteFailed: () => {
+                this.telemetry.metrics.discoveryWriteFailed
+                    .labels({ cache_name: this.name })
+                    .inc();
+            },
+        });
+        await manager.register();
+        return manager;
+    }
     async initAnalyticsSafe() {
         if (this.analyticsInitiated)
             return;
@@ -1056,3 +1307,17 @@ class SemanticCache {
     }
 }
 exports.SemanticCache = SemanticCache;
+// --- Judge helpers ---
+class JudgeTimeoutError extends Error {
+    constructor() {
+        super('judgeFn timed out');
+        this.name = 'JudgeTimeoutError';
+    }
+}
+function raceWithTimeout(p, timeoutMs) {
+    let timer;
+    const timeout = new Promise((_, reject) => {
+        timer = setTimeout(() => reject(new JudgeTimeoutError()), timeoutMs);
+    });
+    return Promise.race([p, timeout]).finally(() => clearTimeout(timer));
+}

package/dist/discovery.d.ts

@@ -0,0 +1,67 @@
+import type { Valkey } from './types';
+export declare const PROTOCOL_VERSION = 1;
+export declare const REGISTRY_KEY = "__betterdb:caches";
+export declare const PROTOCOL_KEY = "__betterdb:protocol";
+export declare const HEARTBEAT_KEY_PREFIX = "__betterdb:heartbeat:";
+export declare const DEFAULT_HEARTBEAT_INTERVAL_MS = 30000;
+export declare const HEARTBEAT_TTL_SECONDS = 60;
+export declare const CACHE_TYPE: "semantic_cache";
+export type CacheType = typeof CACHE_TYPE;
+export interface DiscoveryOptions {
+    enabled?: boolean;
+    heartbeatIntervalMs?: number;
+    includeCategories?: boolean;
+}
+export interface MarkerMetadata {
+    type: CacheType;
+    prefix: string;
+    version: string;
+    protocol_version: number;
+    capabilities: string[];
+    stats_key: string;
+    started_at: string;
+    pid?: number;
+    hostname?: string;
+    [extra: string]: unknown;
+}
+export interface BuildSemanticMetadataInput {
+    name: string;
+    version: string;
+    defaultThreshold: number;
+    categoryThresholds: Record<string, number>;
+    uncertaintyBand: number;
+    includeCategories: boolean;
+}
+export declare function buildSemanticMetadata(input: BuildSemanticMetadataInput): MarkerMetadata;
+export interface DiscoveryLogger {
+    warn: (msg: string) => void;
+    debug: (msg: string) => void;
+}
+export interface DiscoveryManagerDeps {
+    client: Valkey;
+    name: string;
+    metadata: MarkerMetadata;
+    heartbeatIntervalMs?: number;
+    logger?: DiscoveryLogger;
+    onWriteFailed?: () => void;
+}
+export declare class DiscoveryManager {
+    private readonly client;
+    private readonly name;
+    private readonly metadata;
+    private readonly heartbeatIntervalMs;
+    private readonly heartbeatKey;
+    private readonly logger;
+    private readonly onWriteFailed;
+    private heartbeatHandle;
+    constructor(deps: DiscoveryManagerDeps);
+    register(): Promise<void>;
+    stop(opts: {
+        deleteHeartbeat: boolean;
+    }): Promise<void>;
+    tickHeartbeat(): Promise<void>;
+    private startHeartbeat;
+    private safeHget;
+    private safeCall;
+    private checkCollision;
+}

package/dist/discovery.js

@@ -0,0 +1,140 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.DiscoveryManager = exports.CACHE_TYPE = exports.HEARTBEAT_TTL_SECONDS = exports.DEFAULT_HEARTBEAT_INTERVAL_MS = exports.HEARTBEAT_KEY_PREFIX = exports.PROTOCOL_KEY = exports.REGISTRY_KEY = exports.PROTOCOL_VERSION = void 0;
+exports.buildSemanticMetadata = buildSemanticMetadata;
+const node_os_1 = require("node:os");
+const errors_1 = require("./errors");
+exports.PROTOCOL_VERSION = 1;
+exports.REGISTRY_KEY = '__betterdb:caches';
+exports.PROTOCOL_KEY = '__betterdb:protocol';
+exports.HEARTBEAT_KEY_PREFIX = '__betterdb:heartbeat:';
+exports.DEFAULT_HEARTBEAT_INTERVAL_MS = 30_000;
+exports.HEARTBEAT_TTL_SECONDS = 60;
+exports.CACHE_TYPE = 'semantic_cache';
+function buildSemanticMetadata(input) {
+    const metadata = {
+        type: exports.CACHE_TYPE,
+        prefix: input.name,
+        version: input.version,
+        protocol_version: exports.PROTOCOL_VERSION,
+        capabilities: ['invalidate', 'similarity_distribution', 'threshold_adjust'],
+        index_name: `${input.name}:idx`,
+        stats_key: `${input.name}:__stats`,
+        config_key: `${input.name}:__config`,
+        default_threshold: input.defaultThreshold,
+        uncertainty_band: input.uncertaintyBand,
+        started_at: new Date().toISOString(),
+        pid: process.pid,
+        hostname: (0, node_os_1.hostname)(),
+    };
+    if (input.includeCategories && Object.keys(input.categoryThresholds).length > 0) {
+        metadata.category_thresholds = { ...input.categoryThresholds };
+    }
+    return metadata;
+}
+const noopLogger = {
+    warn: () => { },
+    debug: () => { },
+};
+function errMsg(err) {
+    return err instanceof Error ? err.message : String(err);
+}
+class DiscoveryManager {
+    client;
+    name;
+    metadata;
+    heartbeatIntervalMs;
+    heartbeatKey;
+    logger;
+    onWriteFailed;
+    heartbeatHandle = null;
+    constructor(deps) {
+        this.client = deps.client;
+        this.name = deps.name;
+        this.metadata = deps.metadata;
+        this.heartbeatIntervalMs = deps.heartbeatIntervalMs ?? exports.DEFAULT_HEARTBEAT_INTERVAL_MS;
+        this.heartbeatKey = `${exports.HEARTBEAT_KEY_PREFIX}${deps.name}`;
+        this.logger = deps.logger ?? noopLogger;
+        this.onWriteFailed = deps.onWriteFailed ?? (() => { });
+    }
+    async register() {
+        const existingJson = await this.safeHget();
+        if (existingJson !== null) {
+            this.checkCollision(existingJson);
+        }
+        await this.safeCall(() => this.client.hset(exports.REGISTRY_KEY, this.name, JSON.stringify(this.metadata)), 'HSET registry');
+        await this.safeCall(() => this.client.set(exports.PROTOCOL_KEY, String(exports.PROTOCOL_VERSION), 'NX'), 'SET protocol');
+        await this.tickHeartbeat();
+        this.startHeartbeat();
+    }
+    async stop(opts) {
+        if (this.heartbeatHandle) {
+            clearInterval(this.heartbeatHandle);
+            this.heartbeatHandle = null;
+        }
+        if (!opts.deleteHeartbeat) {
+            return;
+        }
+        try {
+            await this.client.del(this.heartbeatKey);
+        }
+        catch (err) {
+            this.logger.debug(`discovery: DEL heartbeat failed: ${errMsg(err)}`);
+        }
+    }
+    async tickHeartbeat() {
+        const now = new Date().toISOString();
+        try {
+            await this.client.set(this.heartbeatKey, now, 'EX', exports.HEARTBEAT_TTL_SECONDS);
+        }
+        catch (err) {
+            this.logger.debug(`discovery: heartbeat SET failed: ${errMsg(err)}`);
+            this.onWriteFailed();
+        }
+    }
+    startHeartbeat() {
+        if (this.heartbeatHandle) {
+            clearInterval(this.heartbeatHandle);
+        }
+        const handle = setInterval(() => {
+            void this.tickHeartbeat();
+        }, this.heartbeatIntervalMs);
+        handle.unref?.();
+        this.heartbeatHandle = handle;
+    }
+    async safeHget() {
+        try {
+            return await this.client.hget(exports.REGISTRY_KEY, this.name);
+        }
+        catch (err) {
+            this.logger.warn(`discovery: HGET registry failed: ${errMsg(err)}`);
+            this.onWriteFailed();
+            return null;
+        }
+    }
+    async safeCall(fn, label) {
+        try {
+            await fn();
+        }
+        catch (err) {
+            this.logger.warn(`discovery: ${label} failed: ${errMsg(err)}`);
+            this.onWriteFailed();
+        }
+    }
+    checkCollision(existingJson) {
+        let parsed;
+        try {
+            parsed = JSON.parse(existingJson);
+        }
+        catch {
+            return;
+        }
+        if (parsed.type && parsed.type !== exports.CACHE_TYPE) {
+            throw new errors_1.SemanticCacheUsageError(`cache name collision: '${this.name}' is already registered as type '${String(parsed.type)}' on this Valkey instance`);
+        }
+        if (parsed.version && parsed.version !== this.metadata.version) {
+            this.logger.warn(`discovery: overwriting marker for '${this.name}' (existing version ${String(parsed.version)}, this version ${this.metadata.version})`);
+        }
+    }
+}
+exports.DiscoveryManager = DiscoveryManager;

package/dist/index.d.ts

@@ -1,8 +1,10 @@
 export { SemanticCache } from './SemanticCache';
 export type { ThresholdEffectivenessResult } from './SemanticCache';
 export { DEFAULT_COST_TABLE } from './defaultCostTable';
-export type { SemanticCacheOptions, CacheCheckOptions, CacheStoreOptions, CacheCheckResult, CacheStats, IndexInfo, InvalidateResult, CacheConfidence, EmbedFn, ModelCost, RerankOptions, } from './types';
+export type { SemanticCacheOptions, CacheCheckOptions, CacheStoreOptions, CacheCheckResult, CacheStats, IndexInfo, InvalidateResult, CacheConfidence, EmbedFn, ModelCost, RerankOptions, JudgeOptions, ConfigRefreshOptions, } from './types';
 export { SemanticCacheUsageError, EmbeddingError, ValkeyCommandError, } from './errors';
 export type { ContentBlock, TextBlock, BinaryBlock, ToolCallBlock, ToolResultBlock, ReasoningBlock, BlockHints, } from './utils';
+export { escapeTag } from './utils';
 export type { BinaryRef, BinaryNormalizer, NormalizerConfig } from './normalizer';
 export { hashBase64, hashBytes, hashUrl, fetchAndHash, passthrough, composeNormalizer, defaultNormalizer, } from './normalizer';
+export type { DiscoveryOptions } from './discovery';

package/dist/index.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.defaultNormalizer = exports.composeNormalizer = exports.passthrough = exports.fetchAndHash = exports.hashUrl = exports.hashBytes = exports.hashBase64 = exports.ValkeyCommandError = exports.EmbeddingError = exports.SemanticCacheUsageError = exports.DEFAULT_COST_TABLE = exports.SemanticCache = void 0;
+exports.defaultNormalizer = exports.composeNormalizer = exports.passthrough = exports.fetchAndHash = exports.hashUrl = exports.hashBytes = exports.hashBase64 = exports.escapeTag = exports.ValkeyCommandError = exports.EmbeddingError = exports.SemanticCacheUsageError = exports.DEFAULT_COST_TABLE = exports.SemanticCache = void 0;
 var SemanticCache_1 = require("./SemanticCache");
 Object.defineProperty(exports, "SemanticCache", { enumerable: true, get: function () { return SemanticCache_1.SemanticCache; } });
 var defaultCostTable_1 = require("./defaultCostTable");
@@ -9,6 +9,8 @@ var errors_1 = require("./errors");
 Object.defineProperty(exports, "SemanticCacheUsageError", { enumerable: true, get: function () { return errors_1.SemanticCacheUsageError; } });
 Object.defineProperty(exports, "EmbeddingError", { enumerable: true, get: function () { return errors_1.EmbeddingError; } });
 Object.defineProperty(exports, "ValkeyCommandError", { enumerable: true, get: function () { return errors_1.ValkeyCommandError; } });
+var utils_1 = require("./utils");
+Object.defineProperty(exports, "escapeTag", { enumerable: true, get: function () { return utils_1.escapeTag; } });
 var normalizer_1 = require("./normalizer");
 Object.defineProperty(exports, "hashBase64", { enumerable: true, get: function () { return normalizer_1.hashBase64; } });
 Object.defineProperty(exports, "hashBytes", { enumerable: true, get: function () { return normalizer_1.hashBytes; } });

package/dist/telemetry.d.ts

@@ -13,6 +13,10 @@ interface CacheMetrics {
     costSavedTotal: Counter;
     embeddingCacheTotal: Counter;
     staleModelEvictions: Counter;
+    discoveryWriteFailed: Counter;
+    configRefreshFailed: Counter;
+    judgeDecisions: Counter;
+    judgeDuration: Histogram;
 }
 export interface Telemetry {
     tracer: Tracer;

package/dist/telemetry.js

@@ -57,6 +57,27 @@ function createTelemetry(opts) {
         help: 'Entries evicted due to staleAfterModelChange detection',
         labelNames: ['cache_name'],
     });
+    const discoveryWriteFailed = getOrCreateCounter(registry, {
+        name: `${opts.prefix}_discovery_write_failed_total`,
+        help: 'Count of failed discovery-marker writes (best-effort HGET/HSET/SET operations against __betterdb:* keys)',
+        labelNames: ['cache_name'],
+    });
+    const configRefreshFailed = getOrCreateCounter(registry, {
+        name: `${opts.prefix}_config_refresh_failed_total`,
+        help: 'Count of failed periodic config refreshes (HGETALL on __config).',
+        labelNames: ['cache_name'],
+    });
+    const judgeDecisions = getOrCreateCounter(registry, {
+        name: `${opts.prefix}_judge_decisions_total`,
+        help: 'LLM-as-judge decisions for borderline cache hits',
+        labelNames: ['cache_name', 'category', 'decision'],
+    });
+    const judgeDuration = getOrCreateHistogram(registry, {
+        name: `${opts.prefix}_judge_duration_seconds`,
+        help: 'Wall-clock duration of judgeFn invocations',
+        labelNames: ['cache_name', 'category', 'decision'],
+        buckets: [0.05, 0.1, 0.25, 0.5, 1, 2, 5],
+    });
     return {
         tracer,
         metrics: {
@@ -67,6 +88,10 @@ function createTelemetry(opts) {
             costSavedTotal,
             embeddingCacheTotal,
             staleModelEvictions,
+            discoveryWriteFailed,
+            configRefreshFailed,
+            judgeDecisions,
+            judgeDuration,
         },
     };
 }

package/dist/types.d.ts

@@ -1,6 +1,13 @@
 import type Valkey from 'iovalkey';
 import type { Registry } from 'prom-client';
+import type { DiscoveryOptions } from './discovery';
 export type { Valkey };
+export interface ConfigRefreshOptions {
+    /** Enable periodic config refresh from Valkey. Default: true. */
+    enabled?: boolean;
+    /** Refresh interval in milliseconds. Default: 30000. Minimum: 1000. */
+    intervalMs?: number;
+}
 export type EmbedFn = (text: string) => Promise<number[]>;
 export interface ModelCost {
     inputPer1k: number;
@@ -92,6 +99,20 @@ export interface SemanticCacheOptions {
         /** Interval in ms for periodic stats snapshots. Default: 300_000 (5 min). 0 to disable. */
         statsIntervalMs?: number;
     };
+    /**
+     * Discovery-marker protocol controls. See
+     * docs/plans/specs/spec-semantic-cache-discovery-markers.md.
+     * Defaults: enabled=true, heartbeatIntervalMs=30000, includeCategories=true.
+     */
+    discovery?: DiscoveryOptions;
+    /**
+     * Periodic refresh of in-memory threshold config from Valkey.
+     * When enabled, the cache re-reads `{name}:__config` on the configured
+     * interval. Field `threshold` updates `defaultThreshold`; fields named
+     * `threshold:{category}` update `categoryThresholds[category]`.
+     * Defaults: enabled=true, intervalMs=30000.
+     */
+    configRefresh?: ConfigRefreshOptions;
 }
 export interface RerankOptions {
     /**
@@ -108,6 +129,61 @@ export interface RerankOptions {
         similarity: number;
     }>) => Promise<number>;
 }
+/**
+ * LLM-as-judge adjudication for borderline cache hits.
+ *
+ * When set on CacheCheckOptions, a hit whose cosine distance lands in the
+ * uncertainty band (threshold - uncertaintyBand < score <= threshold) is
+ * passed to judgeFn before being returned. The judge accepts (promotes the
+ * hit to confidence: 'high') or rejects (treats it as a miss with
+ * nearestMiss populated).
+ *
+ * The judge is NOT invoked for:
+ *   - high-confidence hits (score <= threshold - uncertaintyBand)
+ *   - misses (score > threshold)
+ *   - the no-candidates case (FT.SEARCH returned zero rows)
+ *
+ * When rerank is also set, the judge runs on the reranked pick, not the
+ * original top-1.
+ */
+export interface JudgeOptions {
+    /**
+     * Function that decides whether a borderline cache hit is acceptable.
+     * Return true to accept (caller receives confidence: 'high').
+     * Return false to reject (caller receives a miss with nearestMiss).
+     *
+     * The function receives the original prompt text (or the resolved text
+     * portion of a multipart prompt), the cached response, the cosine distance,
+     * the effective threshold, and the category if one was supplied to check().
+     */
+    judgeFn: (input: {
+        prompt: string;
+        response: string;
+        similarity: number;
+        threshold: number;
+        category: string | undefined;
+    }) => Promise<boolean>;
+    /**
+     * Behavior when judgeFn throws or exceeds timeoutMs.
+     *   'accept' - return the cached response with confidence: 'uncertain'
+     *              (current pre-judge behavior, fail-open).
+     *   'reject' - treat as a miss (fail-closed).
+     * Default: 'accept'.
+     */
+    onError?: 'accept' | 'reject';
+    /**
+     * Per-call timeout in milliseconds. Default: 2000.
+     * The judge function is raced against this timeout; timeout is treated
+     * the same as a thrown error and routed through onError.
+     *
+     * Note: the underlying promise is not cancelled on timeout — JavaScript has
+     * no built-in cancellation primitive. A real LLM HTTP request will continue
+     * running in the background after the timeout fires, consuming API quota.
+     * To stop the underlying request, use an AbortController inside judgeFn and
+     * abort it when the signal you manage fires.
+     */
+    timeoutMs?: number;
+}
 export interface CacheCheckOptions {
     /** Per-request threshold override (cosine distance 0-2). Highest priority. */
     threshold?: number;
@@ -146,6 +222,11 @@ export interface CacheCheckOptions {
      * in rerankFn yourself.
      */
     rerank?: RerankOptions;
+    /**
+     * Optional LLM-as-judge adjudication for borderline hits.
+     * See JudgeOptions. Ignored on checkBatch() - call check() per prompt instead.
+     */
+    judge?: JudgeOptions;
 }
 export interface CacheStoreOptions {
     /** Per-entry TTL in seconds. Overrides SemanticCacheOptions.defaultTtl. */
@@ -202,10 +283,19 @@ export interface CacheCheckResult {
     /**
      * On a miss where a candidate existed but didn't clear the threshold,
      * describes how close it was. Useful for threshold tuning.
+     *
+     * Note: when the miss originates from a judge rejection, `deltaToThreshold`
+     * will be <= 0 because the score did clear the threshold — the judge said no.
+     * Existing non-judge misses always produce deltaToThreshold > 0.
+     * Use `deltaToThreshold <= 0` to detect judge-originated misses.
      */
     nearestMiss?: {
         similarity: number;
         deltaToThreshold: number;
+        /** The effective threshold that was applied. Present on judge-rejection misses. */
+        threshold?: number;
+        /** The Valkey key of the entry that was rejected. Present on judge-rejection misses. */
+        matchedKey?: string;
     };
     /**
      * Estimated cost saved (in dollars) by returning this cached result instead of calling the LLM.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@betterdb/semantic-cache",
-  "version": "0.2.0",
+  "version": "0.5.0",
   "description": "Valkey-native semantic cache for LLM applications with built-in OpenTelemetry and Prometheus instrumentation",
   "keywords": [
     "valkey",