@sigrun/recall 0.0.0-stage → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,42 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [0.1.3] - 2026-11-03
8
+
9
+ ### Added
10
+
11
+ - The search route accepts `?exclude=<conversationId>` so callers can reproduce auto-recall's exclusion of the current conversation (eval fidelity).
12
+ - The eval harness (`scripts/recall-eval.ts`) supports a per-pair `exclude` field with client-side filtering, and over-fetches when excluding so filtered hits don't consume the top-k window.
13
+
14
+ ### Fixed
15
+
16
+ - (from polish wave 1, 0.1.2) The shutdown drain is bounded and its race timer clears on fast settles.
17
+
18
+ ## [0.1.2] - Unreleased
19
+
20
+ ### Fixed
21
+
22
+ - Bound the shutdown drain: each round races `Promise.allSettled(pending)` against a timer for the remaining budget, so a capture whose embedding fetch never settles can no longer hang `harnessShutdown` past the 30s deadline; the database still closes.
23
+
24
+ ## [0.1.1] - Unreleased
25
+
26
+ ### Fixed
27
+
28
+ - Version bump only: 0.1.0 was occupied by a registry-side staging artifact and is permanently unavailable; this is the initial published release.
29
+
30
+ ## [0.1.0] - Unreleased
31
+
32
+ ### Changed (release pinning)
33
+
34
+ - `@sigrun/shared` is pinned as a peer at `0.5.0` (runtime VALUE imports — `accessOf` — ride the peer, with the dev `file:` link to core's workspace, the ext-gateway family posture).
35
+ - `@sigrun/sigrun` stays the development `file:../sigrun` link (types-only consumption; core 0.5.0 carries C1 on the registry — but this repo never imports it at runtime, so no pin is needed for installs).
36
+
37
+ ### Added
38
+
39
+ - Idempotent capture of conversation turns, tool calls/results, and whole large tool results with redacted porter/trigram indexes.
40
+ - Live audience and confidentiality enforcement, optional semantic embeddings, and bounded recency-ranked search.
41
+ - Replay-safe `recall_search`/`recall_open` tools and byte-stable request-only automatic recall.
42
+ - Rotating, resumable backfill cron task plus user search and admin sweep routes.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Michael MacLeod
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,46 @@
1
- # Temporary Holding Version
1
+ # @sigrun/recall
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Cross-conversation recall for [sigrun](https://github.com/sigrun-ai/sigrun), as an optional pi-durable extension.
4
+
5
+ Recall indexes conversation turns, tool calls/results, and complete large tool results in an extension-scoped SQLite database. Settled runs are captured incrementally; a ten-minute backfill job also visits imported or interrupted conversations. Search combines porter stemming, trigram substring matching, optional embeddings, and recency. The model can use `recall_search` and `recall_open`; automatic recall adds bounded, clearly marked reference material to new user messages.
6
+
7
+ ## Settings and secrets
8
+
9
+ Extension settings support `externalizeBytes` (default `24576`) and an optional `embedding` object:
10
+
11
+ ```json
12
+ {
13
+ "externalizeBytes": 24576,
14
+ "embedding": {
15
+ "url": "https://example.invalid/embed",
16
+ "model": "embed-model",
17
+ "local": false,
18
+ "timeoutMs": 800
19
+ }
20
+ }
21
+ ```
22
+
23
+ The optional `embedding-api-key` secret authorizes the `[OI]`-compatible embeddings endpoint. Embeddings are sent in batches and only when configured; failures leave lexical search available. Per-user settings support `autoRecall` (default `true`) and `localProviders` (default `[]`).
24
+
25
+ ## Access and confidentiality
26
+
27
+ Items are `normal` or `local-only`. Local-only material reaches only models whose provider is in the item owner's `localProviders`; an absent model or unknown owner fails closed. Recall also re-checks live conversation sharing for every reader of the calling conversation: material shared later becomes visible, and revoked sharing stops matching immediately. Internal conversations are never indexed or recalled. Displaying local-only material taints subsequent capture in the calling conversation.
28
+
29
+ Auto-recall excludes the current conversation and job runs; explicit `recall_search` can include both. Stored injection blocks are reused byte-for-byte where policy allows and are kept for the conversation's life (they are small; v1 does not prune them).
30
+
31
+ ## Large results and reading
32
+
33
+ Complete tool results larger than `externalizeBytes` are stored whole under `artifacts/`, indexed in bounded passages, and replaced in model output by a head, outline, and handle. Truncated or spilled shell output is not externalized because the hook no longer has the complete result; complete shell output over the threshold is externalized. `recall_open` accepts `{offset, length}` for byte-based artifact slices; the spec's `section` option is deferred.
34
+
35
+ Trigram search cannot match CJK substrings shorter than three characters. Use at least three characters for those queries.
36
+
37
+ ## Development and uninstall
38
+
39
+ ```sh
40
+ npm install
41
+ npm test
42
+ npm run typecheck
43
+ npm run build
44
+ ```
45
+
46
+ `dist/` is committed by design for git-source installs. Tests use synthetic data and do not contact external services. To uninstall recall data, remove the extension and delete `<data>/recall.db` plus `<data>/artifacts/`.
@@ -0,0 +1,32 @@
1
+ import type { ToolExecutionResult, ToolHookCall } from "@earendil-works/pi-durable";
2
+ import type { RecallDb } from "./db.js";
3
+ export interface ArtifactStore {
4
+ put(text: string, meta: {
5
+ title?: string;
6
+ url?: string;
7
+ conversationId: number;
8
+ owner: string | null;
9
+ at: number;
10
+ }): {
11
+ handle: string;
12
+ bytes: number;
13
+ };
14
+ read(handle: string, offset: number, length: number): string | undefined;
15
+ }
16
+ /** Store whole tool results by content hash and index their searchable text. */
17
+ export declare function createArtifactStore(db: RecallDb, dir: string): ArtifactStore;
18
+ /** Return markdown headings, or fixed-size part markers for unheaded text. */
19
+ export declare function outline(text: string): {
20
+ title: string;
21
+ offset: number;
22
+ }[];
23
+ /** Render the retained head and bounded navigation stub for an artifact. */
24
+ export declare function renderStub(text: string, handle: string, headBytes: 6144): string;
25
+ /** Replace only large, complete top-level tool output with an artifact stub. */
26
+ export declare function externalize(result: ToolExecutionResult, call: ToolHookCall, deps: {
27
+ store: ArtifactStore;
28
+ thresholdBytes: number;
29
+ conversationId: number;
30
+ owner: string | null;
31
+ now: number;
32
+ }): ToolExecutionResult | undefined;
@@ -0,0 +1,137 @@
1
+ import { createHash } from "node:crypto";
2
+ import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
3
+ import { join } from "node:path";
4
+ import { redact } from "./redact.js";
5
+ const ARTIFACT_PASSAGE_SIZE = 2000;
6
+ const SYNTHETIC_OUTLINE_SIZE = 4096;
7
+ /** Store whole tool results by content hash and index their searchable text. */
8
+ export function createArtifactStore(db, dir) {
9
+ function pathForHandle(handle) {
10
+ if (!/^a:[0-9a-f]{16}$/.test(handle))
11
+ return undefined;
12
+ const hash = db.artifactHash(handle);
13
+ return hash === undefined ? undefined : join(dir, `${hash}.txt`);
14
+ }
15
+ return {
16
+ put(text, meta) {
17
+ const hash = createHash("sha256").update(text).digest("hex");
18
+ const handle = `a:${hash.slice(0, 16)}`;
19
+ const path = join(dir, `${hash}.txt`);
20
+ if (!existsSync(path)) {
21
+ mkdirSync(dir, { recursive: true });
22
+ const temporary = `${path}.${process.pid}.tmp`;
23
+ writeFileSync(temporary, text, "utf8");
24
+ renameSync(temporary, path);
25
+ }
26
+ db.putArtifact(hash, Buffer.byteLength(text), {
27
+ title: meta.title,
28
+ url: meta.url,
29
+ createdAt: meta.at,
30
+ });
31
+ const buffer = Buffer.from(text, "utf8");
32
+ for (let offset = 0; offset < buffer.length; offset += ARTIFACT_PASSAGE_SIZE) {
33
+ const bytes = buffer.subarray(offset, offset + ARTIFACT_PASSAGE_SIZE);
34
+ const passage = bytes.toString("utf8");
35
+ db.insertItem({
36
+ conversationId: meta.conversationId,
37
+ owner: meta.owner,
38
+ kind: "artifact",
39
+ entryId: null,
40
+ messageIndex: null,
41
+ blockIndex: 0,
42
+ offset,
43
+ length: bytes.length,
44
+ artifact: handle,
45
+ job: false,
46
+ cls: "normal",
47
+ at: meta.at,
48
+ title: meta.title ?? null,
49
+ }, redact(passage));
50
+ }
51
+ return { handle, bytes: Buffer.byteLength(text) };
52
+ },
53
+ read(handle, offset, length) {
54
+ const path = pathForHandle(handle);
55
+ if (path === undefined || !existsSync(path))
56
+ return undefined;
57
+ const buffer = readFileSync(path);
58
+ let start = Math.min(offset, buffer.length);
59
+ let end = Math.min(offset + length, buffer.length);
60
+ while (start > 0 && start < buffer.length && (buffer[start] & 0xc0) === 0x80)
61
+ start -= 1;
62
+ while (end > start && end < buffer.length && (buffer[end] & 0xc0) === 0x80)
63
+ end -= 1;
64
+ return buffer.subarray(start, end).toString("utf8");
65
+ },
66
+ };
67
+ }
68
+ /** Return markdown headings, or fixed-size part markers for unheaded text. */
69
+ export function outline(text) {
70
+ const headings = [];
71
+ for (const match of text.matchAll(/^#{1,6} (.+)$/gm)) {
72
+ headings.push({
73
+ title: match[1],
74
+ offset: Buffer.byteLength(text.slice(0, match.index)),
75
+ });
76
+ }
77
+ if (headings.length > 0)
78
+ return headings;
79
+ const parts = [];
80
+ for (let offset = 0; offset < Buffer.byteLength(text); offset += SYNTHETIC_OUTLINE_SIZE) {
81
+ parts.push({ title: `part ${parts.length + 1}`, offset });
82
+ }
83
+ return parts;
84
+ }
85
+ /** Render the retained head and bounded navigation stub for an artifact. */
86
+ export function renderStub(text, handle, headBytes) {
87
+ const buffer = Buffer.from(text, "utf8");
88
+ const bytes = buffer.length;
89
+ let headBuffer = buffer.subarray(0, headBytes);
90
+ const lastLine = headBuffer.lastIndexOf(0x0a);
91
+ if (lastLine >= 0)
92
+ headBuffer = headBuffer.subarray(0, lastLine + 1);
93
+ const head = headBuffer.toString("utf8");
94
+ const lines = outline(text)
95
+ .slice(0, 40)
96
+ .map(({ title, offset }) => `- ${title} (offset ${offset})`);
97
+ return `${head}\n\n[recall: full result stored as ${handle}, ${bytes} bytes. Outline:]\n${lines.join("\n")}\nRead more with recall_open("${handle}", {offset}).`;
98
+ }
99
+ function textOf(result) {
100
+ return (result.output ?? [])
101
+ .filter((part) => part.type === "text")
102
+ .map((part) => part.text)
103
+ .join("\n");
104
+ }
105
+ function alreadyPartial(result) {
106
+ const structured = result.structuredOutput;
107
+ if (typeof structured !== "object" || structured === null || Array.isArray(structured)) {
108
+ return false;
109
+ }
110
+ const record = structured;
111
+ return record.truncated === true || record.fullOutputPath !== undefined;
112
+ }
113
+ /** Replace only large, complete top-level tool output with an artifact stub. */
114
+ export function externalize(result, call, deps) {
115
+ if (call.parent !== undefined ||
116
+ result.isError === true ||
117
+ alreadyPartial(result) ||
118
+ result.output === undefined) {
119
+ return undefined;
120
+ }
121
+ const text = textOf(result);
122
+ if (Buffer.byteLength(text) <= deps.thresholdBytes)
123
+ return undefined;
124
+ const url = typeof call.arguments.url === "string" ? call.arguments.url : undefined;
125
+ const saved = deps.store.put(text, {
126
+ title: call.name,
127
+ url,
128
+ conversationId: deps.conversationId,
129
+ owner: deps.owner,
130
+ at: deps.now,
131
+ });
132
+ const nonText = result.output.filter((part) => part.type !== "text");
133
+ return {
134
+ ...result,
135
+ output: [...nonText, { type: "text", text: renderStub(text, saved.handle, 6144) }],
136
+ };
137
+ }
@@ -0,0 +1,11 @@
1
+ import { type Sharing } from "@sigrun/shared/access";
2
+ interface ConversationAccess {
3
+ readonly owner?: string;
4
+ readonly sharing?: Sharing;
5
+ }
6
+ /**
7
+ * Return whether an item from `item` may surface to every ordinary reader of
8
+ * `current`. Superuser access is deliberately not part of this decision.
9
+ */
10
+ export declare function audienceCovers(current: ConversationAccess, item: ConversationAccess): boolean;
11
+ export {};
@@ -0,0 +1,36 @@
1
+ import { accessOf } from "@sigrun/shared/access";
2
+ /**
3
+ * Return whether an item from `item` may surface to every ordinary reader of
4
+ * `current`. Superuser access is deliberately not part of this decision.
5
+ */
6
+ export function audienceCovers(current, item) {
7
+ // Ownerless items are readable by everyone.
8
+ if (item.owner === undefined)
9
+ return true;
10
+ if (current.sharing?.scope === "people") {
11
+ const users = new Set(current.sharing.users);
12
+ if (current.owner !== undefined)
13
+ users.add(current.owner);
14
+ for (const user of users) {
15
+ if (accessOf({ user }, item) === "none")
16
+ return false;
17
+ }
18
+ // An ownerless people current with no users has a vacuous user set:
19
+ // only its groups constrain which items may surface.
20
+ // This empty-reader-set behavior is the mechanical table outcome.
21
+ return (current.sharing.groups ?? []).every((group) => item.sharing?.scope === "household" ||
22
+ (item.sharing?.scope === "people" &&
23
+ item.sharing.groups?.includes(group) === true));
24
+ }
25
+ if (current.sharing?.scope === "household") {
26
+ // Level is deliberately ignored: household grants every identified user
27
+ // read access; its level only widens that access to write.
28
+ return item.sharing?.scope === "household";
29
+ }
30
+ if (current.owner === undefined) {
31
+ // A genuinely unshared, ownerless current is readable by everyone.
32
+ return item.sharing?.scope === "household";
33
+ }
34
+ // Private, including an absent sharing grant: only the owner reads current.
35
+ return accessOf({ user: current.owner }, item) !== "none";
36
+ }
@@ -0,0 +1,20 @@
1
+ import { type AnyTask } from "@earendil-works/pi-durable";
2
+ import type { SigrunHost } from "@sigrun/sigrun/extension";
3
+ import type { captureConversation } from "./capture.js";
4
+ import type { RecallDb } from "./db.js";
5
+ export declare const BACKFILL_TASK_NAME = "recall.backfill";
6
+ export interface BackfillDeps {
7
+ host: SigrunHost;
8
+ capture: typeof captureConversation;
9
+ db: RecallDb;
10
+ vectors?: () => Promise<void>;
11
+ now?: () => number;
12
+ entryBudget?: number;
13
+ }
14
+ /** Sweep conversations in rotating id order under a bounded entry/time budget. */
15
+ export declare function runSweep(deps: BackfillDeps): Promise<{
16
+ indexed: number;
17
+ sweepAfter: number;
18
+ }>;
19
+ /** Register the one-phase cron backfill task. */
20
+ export declare function createBackfillTask(deps: BackfillDeps | (() => BackfillDeps)): AnyTask;
@@ -0,0 +1,64 @@
1
+ import { defineTask } from "@earendil-works/pi-durable";
2
+ export const BACKFILL_TASK_NAME = "recall.backfill";
3
+ /** Sweep conversations in rotating id order under a bounded entry/time budget. */
4
+ export async function runSweep(deps) {
5
+ const started = (deps.now ?? Date.now)();
6
+ const [archived, active] = await Promise.all([
7
+ deps.host.conversations.list({ archived: true }),
8
+ deps.host.conversations.list({ archived: false }),
9
+ ]);
10
+ const ids = [...archived, ...active].map(({ id }) => Number(id)).sort((a, b) => a - b);
11
+ let sweepAfter = Number(deps.db.state("sweepAfter") ?? 0);
12
+ const start = ids.findIndex((id) => id > sweepAfter);
13
+ const ordered = start < 0 ? ids : [...ids.slice(start), ...ids.slice(0, start)];
14
+ let indexed = 0;
15
+ let entries = 0;
16
+ const entryBudget = deps.entryBudget ?? 2000;
17
+ for (const id of ordered) {
18
+ if ((deps.now ?? Date.now)() - started >= 20_000 || entries >= entryBudget)
19
+ break;
20
+ try {
21
+ const result = await deps.capture({
22
+ db: deps.db,
23
+ meta: (conversationId) => deps.host.conversations.meta(conversationId),
24
+ now: deps.now ?? Date.now,
25
+ }, id, async (query, limit, cursor) => {
26
+ const page = await deps.host.conversations.entries(id, query, limit, cursor);
27
+ entries += page.items.length;
28
+ return page;
29
+ });
30
+ indexed += result.indexed;
31
+ sweepAfter = id;
32
+ deps.db.setState("sweepAfter", String(id));
33
+ }
34
+ catch (error) {
35
+ console.error(`recall: backfill failed for conversation ${id}`, error);
36
+ }
37
+ }
38
+ await deps.vectors?.();
39
+ return { indexed, sweepAfter };
40
+ }
41
+ /** Register the one-phase cron backfill task. */
42
+ export function createBackfillTask(deps) {
43
+ const resolve = typeof deps === "function" ? deps : () => deps;
44
+ return defineTask({
45
+ name: BACKFILL_TASK_NAME,
46
+ version: 1,
47
+ initial: () => ({ phase: "sweep" }),
48
+ phases: {
49
+ sweep: async (_task, runtime, context) => {
50
+ const result = await runSweep(resolve());
51
+ await runtime.commit(() => ({
52
+ status: "terminal",
53
+ outcome: { status: "completed", result },
54
+ }), context);
55
+ },
56
+ },
57
+ abort: async (_task, runtime, context) => {
58
+ await runtime.commit(() => ({
59
+ status: "terminal",
60
+ outcome: { status: "failed", error: { message: "aborted" } },
61
+ }), context);
62
+ },
63
+ });
64
+ }
@@ -0,0 +1,22 @@
1
+ import type { Cursor, EntryRecord, Page } from "@earendil-works/pi-durable";
2
+ import type { SessionMeta } from "@sigrun/shared/session-meta";
3
+ import type { RecallDb } from "./db.js";
4
+ import type { Embedder } from "./embed.js";
5
+ import type { VectorIndex } from "./vectors.js";
6
+ export interface CaptureDeps {
7
+ db: RecallDb;
8
+ meta(id: number): Promise<SessionMeta>;
9
+ now(): number;
10
+ /** Optional best-effort semantic indexing; capture never awaits embedding. */
11
+ embedder?: Embedder;
12
+ index?: VectorIndex;
13
+ trackEmbedding?: (work: Promise<void>) => void;
14
+ }
15
+ export type EntryReader = (query: {
16
+ minEntryId?: number;
17
+ order: "ascending";
18
+ }, limit: number, cursor?: Cursor) => Promise<Page<EntryRecord, Cursor>>;
19
+ /** Index newly settled conversation entries, resuming from the saved cursor. */
20
+ export declare function captureConversation(deps: CaptureDeps, conversationId: number, read: EntryReader): Promise<{
21
+ indexed: number;
22
+ }>;
@@ -0,0 +1,80 @@
1
+ import { passagesOf } from "./passages.js";
2
+ import { redact } from "./redact.js";
3
+ const PAGE_SIZE = 200;
4
+ /** Index newly settled conversation entries, resuming from the saved cursor. */
5
+ export async function captureConversation(deps, conversationId, read) {
6
+ const meta = await deps.meta(conversationId);
7
+ if (meta.internal === true)
8
+ return { indexed: 0 };
9
+ const savedCursor = deps.db.cursor(conversationId);
10
+ const query = { order: "ascending" };
11
+ if (savedCursor !== undefined)
12
+ query.minEntryId = savedCursor + 1;
13
+ const taintFrom = deps.db.taintFrom(conversationId);
14
+ let indexed = 0;
15
+ let cursor;
16
+ do {
17
+ const page = await read(query, PAGE_SIZE, cursor);
18
+ const insertedForEmbedding = [];
19
+ for (const entry of page.items) {
20
+ if (entry.conversationId !== conversationId)
21
+ continue;
22
+ for (const passage of passagesOf(entry)) {
23
+ const inserted = deps.db.insertItem({
24
+ conversationId,
25
+ owner: meta.owner ?? null,
26
+ kind: passage.kind,
27
+ entryId: passage.entryId,
28
+ messageIndex: passage.messageIndex,
29
+ blockIndex: passage.blockIndex,
30
+ offset: passage.offset,
31
+ length: passage.length,
32
+ artifact: null,
33
+ job: meta.jobId !== undefined,
34
+ cls: taintFrom !== undefined && passage.entryId >= taintFrom
35
+ ? "local-only"
36
+ : "normal",
37
+ at: passage.at,
38
+ title: meta.name ?? meta.autoName ?? null,
39
+ }, redact(passage.text));
40
+ if (inserted !== undefined) {
41
+ indexed += 1;
42
+ insertedForEmbedding.push({
43
+ id: inserted,
44
+ text: redact(passage.text),
45
+ localOnly: taintFrom !== undefined && passage.entryId >= taintFrom,
46
+ });
47
+ }
48
+ }
49
+ }
50
+ if (deps.embedder !== undefined && deps.index !== undefined) {
51
+ const work = embedPage(deps.embedder, deps.index, insertedForEmbedding);
52
+ if (deps.trackEmbedding === undefined)
53
+ void work;
54
+ else
55
+ deps.trackEmbedding(work);
56
+ }
57
+ const lastEntry = page.items.at(-1);
58
+ if (lastEntry !== undefined)
59
+ deps.db.setCursor(conversationId, lastEntry.id);
60
+ cursor = page.next;
61
+ } while (cursor !== undefined);
62
+ return { indexed };
63
+ }
64
+ async function embedPage(embedder, index, inserted) {
65
+ const candidates = inserted.filter(({ localOnly }) => !localOnly || embedder.local);
66
+ for (let offset = 0; offset < candidates.length; offset += 64) {
67
+ const batch = candidates.slice(offset, offset + 64);
68
+ try {
69
+ const vectors = await embedder.embed(batch.map(({ text }) => text));
70
+ for (const [indexInBatch, vector] of vectors.entries()) {
71
+ const candidate = batch[indexInBatch];
72
+ if (candidate !== undefined)
73
+ index.upsert(candidate.id, vector);
74
+ }
75
+ }
76
+ catch (error) {
77
+ console.error("recall: passage embedding failed; item remains lexical-only", error);
78
+ }
79
+ }
80
+ }
@@ -0,0 +1,18 @@
1
+ export interface SettingsReader {
2
+ get(): Promise<Record<string, unknown>>;
3
+ }
4
+ export interface RecallSettings {
5
+ externalizeBytes: number;
6
+ embedding?: {
7
+ url: string;
8
+ model: string;
9
+ local: boolean;
10
+ timeoutMs: number;
11
+ };
12
+ }
13
+ export interface UserRecallSettings {
14
+ autoRecall: boolean;
15
+ localProviders: readonly string[];
16
+ }
17
+ export declare function resolveSettings(store: SettingsReader): Promise<RecallSettings>;
18
+ export declare function resolveUserSettings(store: SettingsReader): Promise<UserRecallSettings>;
package/dist/config.js ADDED
@@ -0,0 +1,71 @@
1
+ const DEFAULT_EXTERNALIZE_BYTES = 24_576;
2
+ const DEFAULT_EMBEDDING_TIMEOUT_MS = 800;
3
+ function warnInvalid(setting) {
4
+ console.warn(`recall: invalid ${setting} setting; using its default`);
5
+ }
6
+ function positiveInteger(value, fallback, setting) {
7
+ if (value === undefined)
8
+ return fallback;
9
+ if (typeof value === "number" && Number.isInteger(value) && value > 0)
10
+ return value;
11
+ warnInvalid(setting);
12
+ return fallback;
13
+ }
14
+ function nonEmptyString(value, setting) {
15
+ if (typeof value === "string" && value !== "")
16
+ return value;
17
+ warnInvalid(setting);
18
+ return undefined;
19
+ }
20
+ function embeddingSettings(value) {
21
+ if (value === undefined || value === null)
22
+ return undefined;
23
+ if (typeof value !== "object" || Array.isArray(value)) {
24
+ warnInvalid("embedding");
25
+ return undefined;
26
+ }
27
+ const raw = value;
28
+ const url = nonEmptyString(raw.url, "embedding.url");
29
+ const model = nonEmptyString(raw.model, "embedding.model");
30
+ if (url === undefined || model === undefined)
31
+ return undefined;
32
+ if (raw.local !== undefined && typeof raw.local !== "boolean") {
33
+ warnInvalid("embedding.local");
34
+ }
35
+ return {
36
+ url,
37
+ model,
38
+ local: typeof raw.local === "boolean" ? raw.local : false,
39
+ timeoutMs: positiveInteger(raw.timeoutMs, DEFAULT_EMBEDDING_TIMEOUT_MS, "embedding.timeoutMs"),
40
+ };
41
+ }
42
+ export async function resolveSettings(store) {
43
+ const raw = await store.get();
44
+ const settings = {
45
+ externalizeBytes: positiveInteger(raw.externalizeBytes, DEFAULT_EXTERNALIZE_BYTES, "externalizeBytes"),
46
+ };
47
+ const embedding = embeddingSettings(raw.embedding);
48
+ if (embedding !== undefined)
49
+ settings.embedding = embedding;
50
+ return settings;
51
+ }
52
+ export async function resolveUserSettings(store) {
53
+ const raw = await store.get();
54
+ if (raw.autoRecall !== undefined && typeof raw.autoRecall !== "boolean") {
55
+ warnInvalid("autoRecall");
56
+ }
57
+ let localProviders = [];
58
+ if (raw.localProviders !== undefined) {
59
+ if (Array.isArray(raw.localProviders) &&
60
+ raw.localProviders.every((provider) => typeof provider === "string" && provider !== "")) {
61
+ localProviders = [...raw.localProviders];
62
+ }
63
+ else {
64
+ warnInvalid("localProviders");
65
+ }
66
+ }
67
+ return {
68
+ autoRecall: typeof raw.autoRecall === "boolean" ? raw.autoRecall : true,
69
+ localProviders,
70
+ };
71
+ }