@4fu/arbor-notes-sqlite 2026.1106.0 → 2026.1107.0

package/README.md

@@ -27,9 +27,11 @@ everything that takes judgment, such as git, merges and conflicts, to the agent.
   guarantees what an agent cannot guarantee for itself - the record, isolation,
   exactly-once effects, the checks, dispatch. Everything that takes judgment,
   such as git, merges and conflicts, is the agent's.
-- **A directory per window.** Each window gets its own directory, and a
-  continuation keeps working in its predecessor's directory in place. It is not
-  bound to a repository; the agent clones and pushes what it needs.
+- **A shared workspace with per-window checkouts.** The default workspace is
+  ovnfs: one shared folder every window reads, and a window opens its own
+  copy-on-write checkout of the project it needs, works there and merges back
+  with `apply`. It falls back to a plain directory per window when ovnfs is not
+  installed. It is not a sandbox: ovnfs isolates files, not processes.
 - **A self-changing environment.** The model adds and changes skills, MCP
   servers and resident services itself; a `reload` validates the change and
   broadcasts it to every live window as a delta of names and descriptions.
@@ -54,7 +56,9 @@ npm install @4fu/arbor-core @4fu/arbor-journal-sqlite @4fu/arbor-notes-sqlite \
 ```
 
 `@4fu/arbor-plugin-http` is optional; it is the example plugin below.
-`@4fu/arbor-segmenter-jieba` is an optional peer of the journal.
+`@4fu/arbor-segmenter-jieba` is an optional peer of the lexical package, and
+`@4fu/arbor-search-zvec` an optional peer of the journal: absent either one, the
+journal searches lexically and nothing else changes.
 
 ## A minimal host
 
@@ -118,7 +122,10 @@ window gets, and how a continuation keeps working in it, are in
 | [`@4fu/arbor-journal-sqlite`](packages/journal-sqlite) | the default `Journal` and `BlobStore` on SQLite |
 | [`@4fu/arbor-notes-sqlite`](packages/notes-sqlite) | the default `NoteStore` on SQLite, beside the journal |
 | [`@4fu/arbor-segmenter-jieba`](packages/segmenter-jieba) | the default `Segmenter`: jieba's dictionary segmentation for the word index, optional |
-| [`@4fu/arbor-workspace-dir`](packages/workspace-dir) | the default `Workspace`: one directory per window, with a continuation working in its predecessor's directory in place |
+| [`@4fu/arbor-lexical`](packages/lexical) | the lexical search the journal and notes share: word segmentation, the FTS5 helpers and the merge order |
+| [`@4fu/arbor-search-zvec`](packages/search-zvec) | the optional semantic channel: event text embedded and searched by meaning with zvec |
+| [`@4fu/arbor-workspace-dir`](packages/workspace-dir) | the fallback `Workspace`: one directory per window, with a continuation working in its predecessor's directory in place |
+| [`@4fu/arbor-workspace-ovnfs`](packages/workspace-ovnfs) | the default `Workspace` when ovnfs is installed: one shared folder, per-window copy-on-write checkouts the model opens and applies back |
 | [`@4fu/arbor-environment-dir`](packages/environment-dir) | the default `Environment`: one `/env` directory, a revision per item, validation and deltas |
 | [`@4fu/arbor-supervisor-process`](packages/supervisor-process) | the default `Supervisor`: one host process per service, restart with backoff, logs |
 | [`@4fu/arbor-judge-jev`](packages/judge-jev) | the default `Judge`: Jev, TypeSafe AI's System One classifier, answering the dispatch decisions |
@@ -151,7 +158,7 @@ deployment can see what it would be taking on before it adopts one.
 | [docs/coordination.md](docs/coordination.md) | the board, expectations, deliveries and broadcasts, staying current, effects |
 | [docs/environment.md](docs/environment.md) | skills, MCP servers and services: using, changing, reload, built-in skills, deltas |
 | [docs/memory.md](docs/memory.md) | notes, the active set, anchors, links |
-| [docs/workspaces.md](docs/workspaces.md) | per-window directories: isolation, continuity, what the agent does |
+| [docs/workspaces.md](docs/workspaces.md) | the shared folder and per-window checkouts: isolation, continuity, what the agent does |
 | [docs/tools.md](docs/tools.md) | direct tools and code mode modules |
 | [docs/prompting.md](docs/prompting.md) | how every model-facing text is written: layers, notes, deliveries, errors, judge questions, terms |
 | [docs/release.md](docs/release.md) | calendar versioning, the manual release steps, how a package is published |
@@ -168,6 +175,7 @@ tarball would contain ([docs/release.md](docs/release.md)).
 | `packages/errors` | the error shape every refusal uses: the rule and the next action |
 | `packages/journal-sqlite` | the default `Journal` and `BlobStore` on SQLite |
 | `packages/segmenter-jieba` | the default `Segmenter`: jieba's dictionary segmentation for the word index, optional |
+| `packages/lexical` | the lexical search the journal and notes share: word segmentation, the FTS5 helpers and the merge order |
 | `packages/notes-sqlite` | the default `NoteStore` on SQLite, beside the journal |
 | `packages/judge-jev` | the default `Judge`: Jev, TypeSafe AI's System One classifier, answering the dispatch decisions |
 | `packages/workspace-dir` | the default `Workspace`: one directory per window, with a continuation working in its predecessor's directory in place, and the lookup of another window's directory it contributes to the catalog |

package/index.d.ts

@@ -4,10 +4,12 @@
 export { SqliteNoteStore, SUMMARY_LIMIT, DEFAULT_BODY_LIMIT, MAX_BODY_LIMIT } from "./notes.ts";
 export { DEFAULT_MIN_INTERVAL_MS, DEFAULT_ACTIVE_BUDGET, DEFAULT_JUDGE_SHARE, DEFAULT_HALF_LIFE_MS, deriveSummary, } from "./notes.ts";
 export type { SqliteNoteStoreOptions } from "./notes.ts";
-export { SCHEMA_VERSION, FTS_TABLE, createSchema } from "./schema.ts";
+export { SCHEMA_VERSION, FTS_TABLE, WORD_TABLE, createSchema } from "./schema.ts";
 export { noteIdOf, noteRefOf, parseNoteId, parseNoteRef, noteRefForms, storedWindowRef, storedEventRef, } from "./refs.ts";
+export { fallbackSegmenter, resolveSegmenter, indexText } from "@4fu/arbor-lexical";
 export { USE_WEIGHTS, MAX_CREDIT_HOPS, decay, estimateTokens, linkedCredit, loadCredit, loadEdges, pickByBudget, applyJudgeShare, scoreNotes, } from "./score.ts";
 export type { ScoredNote } from "./score.ts";
 export { MIN_TRIGRAM_CHARS, substringClause, holdingIds, escapeLike, likePattern, quoteFts, } from "./search.ts";
 export type { SubstringClause } from "./search.ts";
 export { bodyDiff } from "./diff.ts";
+export { resolveSemanticIndex } from "./semantic.ts";

package/index.js

@@ -3,8 +3,10 @@
  */
 export { SqliteNoteStore, SUMMARY_LIMIT, DEFAULT_BODY_LIMIT, MAX_BODY_LIMIT } from "./notes.js";
 export { DEFAULT_MIN_INTERVAL_MS, DEFAULT_ACTIVE_BUDGET, DEFAULT_JUDGE_SHARE, DEFAULT_HALF_LIFE_MS, deriveSummary, } from "./notes.js";
-export { SCHEMA_VERSION, FTS_TABLE, createSchema } from "./schema.js";
+export { SCHEMA_VERSION, FTS_TABLE, WORD_TABLE, createSchema } from "./schema.js";
 export { noteIdOf, noteRefOf, parseNoteId, parseNoteRef, noteRefForms, storedWindowRef, storedEventRef, } from "./refs.js";
+export { fallbackSegmenter, resolveSegmenter, indexText } from "@4fu/arbor-lexical";
 export { USE_WEIGHTS, MAX_CREDIT_HOPS, decay, estimateTokens, linkedCredit, loadCredit, loadEdges, pickByBudget, applyJudgeShare, scoreNotes, } from "./score.js";
 export { MIN_TRIGRAM_CHARS, substringClause, holdingIds, escapeLike, likePattern, quoteFts, } from "./search.js";
 export { bodyDiff } from "./diff.js";
+export { resolveSemanticIndex } from "./semantic.js";

package/notes.d.ts

@@ -15,7 +15,11 @@
  *   count and token budget, and frozen with the epoch's version, so every
  *   window opened in one epoch has the same prefix and later windows see the
  *   changes as a delta;
- * - every write recorded in the journal as a `note_revision` event.
+ * - every write recorded in the journal as a `note_revision` event;
+ * - lexical search: a word index with BM25, a trigram index for exact
+ *   substrings, and a `LIKE` scan for one- and two-character queries, merged
+ *   with exact substring hits first - and a semantic channel, note text embedded
+ *   and searched by meaning, when `@4fu/arbor-search-zvec` is installed.
  *
  * It is synchronous inside: one `DatabaseSync`, one transaction per write, so
  * the revision a write checks and the revision it writes are one atomic step.
@@ -23,7 +27,7 @@
  * database and the journal. The interface is asynchronous because a `NoteStore`
  * may live behind a process boundary.
  */
-import type { ActiveCandidate, ActiveSetBudget, Journal, Note, NoteBody, NoteContext, NoteDelta, NoteQuery, NoteRef, NoteRevision, NoteSummary, NoteUseInput, NoteWriteInput, NotesSnapshot, NoteStore, Page, SnapshotOptions } from "@4fu/arbor-interfaces";
+import type { ActiveCandidate, ActiveSetBudget, Journal, Note, NoteBody, NoteContext, NoteDelta, NoteQuery, NoteRef, NoteRevision, NoteSummary, NoteUseInput, NoteWriteInput, NotesSnapshot, NoteStore, Page, Segmenter, SemanticIndex, SnapshotOptions } from "@4fu/arbor-interfaces";
 /** A summary is at most this many characters (docs/memory.md). */
 export declare const SUMMARY_LIMIT = 200;
 /** How many characters of a body one read returns. */
@@ -46,6 +50,22 @@ export interface SqliteNoteStoreOptions {
     now?: () => number;
     /** Usage loses half of its weight this often. 24 hours. */
     usageHalfLifeMs?: number;
+    /** Word segmentation for the word index. Defaults to jieba, or the approximation. */
+    segmenter?: Segmenter;
+    /**
+     * The semantic channel: note text embedded and searched by meaning
+     * (docs/memory.md#search). Omitted, the zvec channel is used when
+     * `@4fu/arbor-search-zvec` is installed and its store opens, and there is no
+     * channel otherwise. `null` forces lexical search even when the package is
+     * installed.
+     */
+    semantic?: SemanticIndex | null;
+    /**
+     * One fixed string from the deployment's text table, so this implementation
+     * writes none of its own (docs/prompting.md#language-and-the-text-table).
+     * Defaults to the framework's own Chinese table.
+     */
+    text?: (key: string, args?: Record<string, string | number>) => string;
 }
 /** The opening of a body, when a write carries no summary of its own. */
 export declare function deriveSummary(body: string): string;
@@ -54,9 +74,17 @@ export declare class SqliteNoteStore implements NoteStore {
     private readonly db;
     private readonly journal;
     private readonly clock;
+    private readonly segmenter;
     private readonly halfLifeMs;
+    private readonly semantic;
+    /** The temporary directory an in-memory store's vector collection lives in. */
+    private readonly ownedVectorDirectory;
+    /** The table every refusal this store raises is written from. */
+    private readonly text;
     /** One write at a time: a write spans the notes database and the journal. */
     private writeQueue;
+    private semanticReady;
+    private semanticCaughtUp;
     private closed;
     constructor(options: SqliteNoteStoreOptions);
     write(input: NoteWriteInput): Promise<Note>;
@@ -92,7 +120,44 @@ export declare class SqliteNoteStore implements NoteStore {
     }): Promise<Page<ActiveCandidate>>;
     recordUse(input: NoteUseInput): Promise<void>;
     close(): Promise<void>;
+    /**
+     * Index the notes the semantic channel does not hold yet, and say how many.
+     *
+     * A restart over an existing database, a channel that was not ready when the
+     * notes were written, and a write whose indexing failed all leave notes to
+     * catch up on. A channel that is not ready yet holds what it was given and
+     * indexes that itself, so this answers 0 until it is.
+     */
+    catchUpSemantic(): Promise<number>;
     private ensureOpen;
+    /**
+     * Hand one note's text to the semantic channel, when there is one.
+     *
+     * The note is embedded as one text: its title, its summary and its body, in
+     * the order the lexical index weights them. The vector channel holds one
+     * vector per note and cannot weight a note's parts against each other the way
+     * BM25 weights the three columns, so the title - the note's name - leads the
+     * text and the body follows it, and a query that means either reaches the
+     * note (docs/memory.md#search).
+     */
+    private addToSemantic;
+    /** Take one note's text back out of the semantic channel, as a delete does. */
+    private removeFromSemantic;
+    /**
+     * The semantic channel's candidates, as ranks in the order it gave them.
+     *
+     * An absent channel, one that is not ready yet, and one that refuses all
+     * answer with nothing, and the search is exactly the lexical one. The note
+     * read's own filters decide which candidates count, so a channel that filters
+     * less than the store does still answers the read it was asked for.
+     */
+    private semanticRanks;
+    /**
+     * The notes among `ids` the read's own filters allow, in the order given. A
+     * deleted note is not among them, so a vector that outlived its note never
+     * reaches a page.
+     */
+    private allowedNoteIds;
     private noteRow;
     private revisionRow;
     private latestRevisionRow;
@@ -100,6 +165,8 @@ export declare class SqliteNoteStore implements NoteStore {
     /** The current summary and revision of several notes, in one read. */
     private noteStatesFor;
     private summaryPageBySql;
+    /** One page of an ordered id list, in the order the list gives. */
+    private summaryPage;
     private summaryPageByIds;
     /**
      * The notes about a target, or linked to a ref: the notes the events on that
@@ -121,6 +188,8 @@ export declare class SqliteNoteStore implements NoteStore {
      * the changes since reach it as a delta.
      */
     private epochAt;
+    /** What the active notes of one epoch cost, as the epoch froze them. */
+    private epochTokens;
     /** The active notes one epoch froze, by summary, sorted by id. */
     private epochSummaries;
 }