@andespindola/brainlink 0.1.0-beta.15 → 0.1.0-beta.150

package/AGENTS.md

@@ -83,6 +83,9 @@ Use watch mode while editing notes:
 ```bash
 npm run dev -- server --vault ./vault --watch
 npm run dev -- watch --vault ./vault
+npm run dev -- bench --vault ./vault
+npm run dev -- bench --vault ./vault --watch
+npm run dev -- pack-backup --vault ./vault
 ```
 
 Start MCP over stdio:

package/CHANGELOG.md

@@ -22,6 +22,30 @@
 - Added short-lived hybrid search cache with automatic invalidation on index changes.
 - Added `stats --extended` observability output with storage, quality and latency probes.
 - Added `docs/QUICKSTART.md` and aligned README/agent docs with the latest CLI/MCP flows.
+- Added middle-out context assembly so chunk selection expands around the strongest note chunk.
+- Added compressed-space pack prefiltering (token bloom index) before `.blpk` decryption and scan.
+- Improved graph UI auto-fit and viewport recovery so loaded nodes are re-centered when zoom/pan drifts to empty canvas.
+- Added cross-platform native desktop GUI auto-open for `blink server` (macOS Swift/WebKit, Windows PowerShell WinForms, Linux Python GTK/WebKit2), with app-window/browser fallback.
+- Changed Linux default UI launch to app-window/browser for lighter startup; Linux native GUI is now opt-in via `BRAINLINK_LINUX_NATIVE_GUI=1`.
+- Added native GUI parent-process monitoring so GUI windows close automatically when `blink server` stops.
+- Improved non-mac browser detection fallback to try installed Edge/Chrome/Firefox/Chromium candidates before system default open.
+- Improved graph filter rendering to keep hub anchor nodes visible (`Memory Hub`/`MOC`/high-degree fallback) for coherent relationship context.
+- Fixed graph modal content loading by correcting agent query parameter composition for `/api/graph-node` and `/api/graph-filter` requests.
+- Improved 50k+ graph rendering performance with viewport-aware spatial node culling, cached render visibility, and node-adjacent edge selection to avoid full graph scans every frame.
+- Added incremental vault indexing with file snapshots to reuse unchanged documents/chunks/embeddings, plus adaptive search-pack rebuild thresholds to avoid full re-compression on small edits.
+- Reduced large-graph HTTP payload size with compact `/api/graph-layout` encoding for high-node vaults and capped transmitted edges to improve UI load responsiveness.
+- Added aggressive graph LOD clustering when zoomed out, dynamic per-zoom edge render budgets, and a dedicated frontend worker for off-main-thread graph filter matching.
+- Improved Linux browser fallback launch stability by auto-applying Chromium compatibility flags (`--ozone-platform=x11`, `--disable-gpu`, `--disable-features=Vulkan,VaapiVideoDecoder`, `--disable-background-networking`) for app-window/browser modes.
+- Improved massive-graph UI responsiveness with stricter render budgets, adaptive heavy-graph frame throttling, reduced interaction hit-test frequency, and URL-first agent selection on initial graph load.
+- Improved 50k+ graph LOD behavior so zoomed-out views render lightweight cluster overviews and progressively reveal nodes/edges only as zoom increases.
+- Added `blink bench` with realtime index phase telemetry and per-run compressed-pack analysis (input/output bytes, ratio, saved space, rebuild reason and duration), including continuous watch mode.
+- Added tunable single-stage search-pack compression settings (`searchPack.rowChunkSize`, `searchPack.compressionLevel`, `searchPack.useDictionary`).
+- Added benchmark guardrails for compression savings and latency regression (`searchPack.guardrailMinSavingsPercent`, `searchPack.guardrailMaxLatencyRegressionPercent`), reported in `blink bench`.
+- Added `blink pack-backup` for offline second-stage compression backups of encrypted `.blpk` packs, outside the online query path.
+- Hardened Linux browser launch flags for Ubuntu 26 Chromium/Wayland compatibility (`--disable-vulkan`, `--use-gl=swiftshader`, `--ozone-platform-hint=x11`).
+- Improved pack resilience by auto-repairing missing search-pack manifests from existing `.blpk` files, avoiding unnecessary full repacks on small incremental updates.
+- Updated Linux graph auto-open behavior to prioritize the system default browser (`xdg-open`) before explicit browser fallbacks.
+- Removed implicit Chromium dependency in Linux auto-open flow; app-window launch is now opt-in (`BRAINLINK_LINUX_APP_WINDOW=1`).
 
 ## 0.1.0-beta.3
 

package/COPYRIGHT.md

@@ -0,0 +1,5 @@
+Copyright (c) 2026 Substructa
+
+This project is licensed under the MIT License.
+
+See [LICENSE](./LICENSE) for full terms.

package/README.md

@@ -58,6 +58,7 @@ LLMs do not have infinite context. Brainlink gives agents an external memory lay
 
 Markdown is the source of truth. `.brainlink/index.json` is a rebuildable index artifact.
 After each index run, Brainlink also writes private encrypted search packs at `.brainlink/search-packs/*.blpk` to preserve fast retrieval and portable recovery.
+Online retrieval always uses a single compression stage per pack; optional second-stage compression is reserved for offline backup artifacts only.
 Pack decryption uses a Brainlink key from `$BRAINLINK_HOME/keys` or from `BRAINLINK_SEARCH_PACK_KEY` when explicitly configured.
 Legacy `.jsonl.gz` packs are upgraded to `.blpk` automatically on first search/context access.
 
@@ -67,8 +68,13 @@ Legacy `.jsonl.gz` packs are upgraded to `.blpk` automatically on first search/c
 - Obsidian-compatible `[[wiki links]]` and `#tags`.
 - Weighted graph edges so agents can rank relationship importance and priority.
 - Backlinks, broken-link reports, orphan detection and validation.
-- Full-text, semantic and hybrid retrieval modes.
 - Full-text, semantic and hybrid retrieval on a local file index.
+- Middle-out context assembly around the strongest chunk per document.
+- In-process index and context caching with automatic invalidation on index updates.
+- HTTP graph server caches generated frontend assets and graph-layout JSON payloads by signature, and skips layout serialization when ETag returns `304`.
+- Compressed-space prefiltering for `.blpk` packs before decryption and scan.
+- Incremental indexing that reprocesses only changed markdown files and reuses existing chunks/embeddings for unchanged notes.
+- Adaptive compressed-pack rebuild policy to keep indexing fast during small edit batches.
 - Agent namespaces under `agents/<agent-id>/`.
 - S3-compatible bucket vaults through `s3://bucket/prefix` URIs.
 - CLI with machine-readable `--json` output.
@@ -76,6 +82,17 @@ Legacy `.jsonl.gz` packs are upgraded to `.blpk` automatically on first search/c
 - Built-in MCP stdio server for agent tool integration.
 - Local HTTP API.
 - Realtime graph UI with agent selector and colored knowledge groups.
+- Graph renderer uses a star layout centered on the primary hub while preserving real weighted `[[wiki link]]` edges for backlinks, ranking and context traversal.
+- The full filtered graph stays visible during zoom/pan, rendering every visible node and edge without viewport culling or edge caps in the main view.
+- Graph exploration uses viewport-first chunk streaming (`/api/graph-stream`) with explicit node/edge budgets.
+- Render pipeline uses WebGL in a dedicated worker through `OffscreenCanvas`, keeping the main thread focused on UI controls and details panels.
+- Large graph layout API automatically uses compact payload encoding with link-coverage-aware edge selection to reduce initial client load without hiding major relationships.
+- Large-segment layout spacing now grows logarithmically to keep initial visual density consistent between medium and very large vaults (for example, ~1k vs ~50k notes).
+- Graph coordinates are visually compacted across graph sizes so reset starts from a stable fitted scene and zoom-in progressively reveals local detail.
+- Zoomed-out graph keeps the same flat graph scene and preserves complete filtered relationships without switching to nested subgraphs.
+- Graph reset fits the full graph scene instead of starting in a separate macro overview mode.
+- Graph filtering runs in a dedicated browser worker to keep the UI thread responsive during heavy datasets.
+- Node titles are shown as the user zooms closer, while labels remain bounded to visible on-screen nodes in very large graphs.
 
 ## Install
 
@@ -395,6 +412,7 @@ blink agent upgrade
 ```
 
 This configures `~/.codex/config.toml` with Brainlink MCP (`brainlink-mcp`) so Brainlink is available by default in agent sessions.
+`agent install` and `agent upgrade` also apply the MCP `fully-auto` bootstrap policy by default (`enforceBootstrap`, `enforceContextFirst`, `autoBootstrapOnRead`, `autoBootstrapOnStartup` all enabled).
 
 If you are inside this repository and want plugin gallery setup too:
 
@@ -514,8 +532,12 @@ Available tools:
 - `brainlink_recommendations`: return an automatic action plan so agents can run Brainlink in the recommended order.
 - `brainlink_context`: read indexed context for a task or question.
 - `brainlink_search`: search indexed notes.
+- `brainlink_dedupe`: detect duplicate candidates using exact hash + semantic similarity scores.
+- `brainlink_resolve_duplicate`: resolve duplicate pairs (`merge`, `link`, `ignore`) with connectivity-safe fallback edges.
 - `brainlink_add_note`: write durable Markdown memory and reindex.
 - `brainlink_add_file`: ingest a local file as a note and reindex.
+- `brainlink_volatile_add`: write temporary agent-decided memory with TTL; volatile sections are included in context and never create durable graph edges.
+- `brainlink_volatile_clear`: clear temporary memory for the current vault/agent namespace.
 - `brainlink_index`: rebuild the vault index.
 - `brainlink_stats`: read indexed vault statistics.
 - `brainlink_validate`: validate broken links and orphan notes.
@@ -541,7 +563,7 @@ Agents can raise the importance of a relationship by putting priority markers on
 Related: [[Incident Runbook]] #critical
 ```
 
-Indexed edges expose `weight` and `priority` (`low`, `normal`, `high`, `critical`) through CLI JSON, HTTP graph APIs and `brainlink_graph`.
+Indexed edges expose `weight` and `priority` (`low`, `normal`, `high`, `critical`) through CLI JSON, HTTP graph APIs and `brainlink_graph`. Brainlink promotes only representative graph links per note: high-priority and high-weight links win, structural hub links such as `Memory Hub`, `Knowledge Root`, `MOC` and map notes are suppressed when stronger direct links exist, and old indexes are rebuilt automatically when their graph link model version is missing or stale.
 
 ## Graph UI
 
@@ -552,20 +574,41 @@ blink server --host 127.0.0.1 --port 4321 --watch
 ```
 
 By default, the server uses `$HOME/.brainlink/vault`. Pass `--vault ./vault` only when you want to inspect a custom vault.
+By default, `blink server` tries to open the graph in a native desktop GUI window:
+- macOS: Swift + WebKit
+- Windows: PowerShell WinForms WebBrowser
+- Linux: optional Python GTK + WebKit2 (requires `python3` + `gi` + `WebKit2`)
+
+On Linux, native GUI is disabled by default for better startup performance. Enable it with `BRAINLINK_LINUX_NATIVE_GUI=1`.
+If native GUI launch is unavailable on your system, it falls back to dedicated app-window mode and then to the default browser.
+For Chromium-family browsers on Linux (`chromium`, `chromium-browser`, `google-chrome`, `microsoft-edge`, `brave-browser`), Brainlink now auto-applies compatibility flags during launch (`--ozone-platform=x11`, `--ozone-platform-hint=x11`, `--disable-gpu`, `--disable-vulkan`, `--use-gl=swiftshader`, `--disable-features=Vulkan,VaapiVideoDecoder`, `--disable-background-networking`) to avoid common Wayland/Vulkan/VAAPI startup issues.
+On Linux, Brainlink opens the graph through the system default browser first (`xdg-open`), then `$BROWSER`/detected browsers as fallback. Chromium-family app-window mode is optional via `BRAINLINK_LINUX_APP_WINDOW=1`.
+Use `--no-open` to keep it headless.
+When native GUI is used, the GUI window automatically closes when the `blink server` process stops.
 
 The graph UI shows:
 
 - notes as nodes
-- `[[wiki links]]` as weighted edges
-- details opened on node click (tags, outgoing links, backlinks, full Markdown content)
+- representative `[[wiki links]]` as weighted edges
+- star layout centered on the primary hub, without rewriting or flattening underlying relationships
+- details opened in a non-modal side panel (tags, outgoing links, backlinks, full Markdown content), so zoom and pan remain available while inspecting data
 - neutral graph nodes with segment/group metadata
-- agent selector for isolated views
+- agent selector (id-only labels) for isolated views
 - graph filter matches title, path, tags and note content
+- graph filter keeps hub context nodes visible (`Memory Hub`/`MOC`/high-degree fallback) to preserve relationship readability
 - realtime refresh while `--watch` is enabled
 - graph controls for zoom in, zoom out, fit visible nodes and reset-to-fit-all
-- wheel zoom anchored to cursor position for faster navigation in large graphs
+- wheel zoom (including `cmd+scroll` and `ctrl+scroll`) anchored to cursor position for faster navigation in large graphs
+- wheel/button zoom updates immediately at the cursor anchor without delayed focus-transition interpolation
+- Bloom-like scene navigation: reset fits the current graph scene, wheel zoom stays anchored to the cursor, and worker-driven WebGL rendering keeps pan/zoom interaction responsive
+- zoom-out floor for large and massive graphs to keep the scene reachable without switching into a separate macro graph mode
+- keyboard shortcuts: `+` zoom in, `-` zoom out, `0` reset fit
+- click on a node opens its details panel; double-click on empty canvas zooms in at cursor position
 - floating graph totals (notes, links, tags) below the Brainlink title
-- large-graph rendering safeguards (edge draw caps, lower redraw rate, zoom-aware interaction)
+- graph rendering safeguards (batched GPU draw calls, lower redraw rate, zoom-aware interaction)
+- adaptive CPU safeguards for large graphs: idle frame pacing, throttled background physics updates and cached viewport dimensions to reduce redraw/layout overhead while preserving interaction responsiveness
+- worker-first WebGL rendering with Canvas fallback when `OffscreenCanvas` or worker rendering is unavailable
+- large graph view keeps a single-level graph model across zoom levels, renders the full filtered scene instead of viewport-sampled subsets, and shows node titles as zoom approaches readable scale
 
 The server indexes before starting by default. Use `--no-index` to skip that step:
 
@@ -584,6 +627,8 @@ Routes:
 - `GET /api/agents`
 - `GET /api/graph`
 - `GET /api/graph-layout`
+- `GET /api/graph-view?x=<x>&y=<y>&w=<width>&h=<height>&scale=<scale>`
+- `GET /api/graph-stream?x=<x>&y=<y>&w=<width>&h=<height>&scale=<scale>&nodeBudget=<n>&edgeBudget=<n>`
 - `GET /api/graph-node?id=<node-id>`
 - `GET /api/search?q=<query>&limit=10&mode=hybrid`
 - `GET /api/context?q=<query>&limit=12&tokens=2000&mode=hybrid`
@@ -652,6 +697,7 @@ blink config set-vault "s3://my-memory-bucket/brainlink" --global
 
 `config set-vault` writes configuration through CLI (no manual file edits required).  
 By default it writes local config (`./brainlink.config.json`), appends the vault to `allowedVaults`, and migrates Markdown memory from the current configured vault when the target is empty.  
+When the configured default vault is changed manually in config files, Brainlink also performs automatic migration on the next command that uses the configured vault (without explicit `--vault`).
 Use `--global` to write to `$BRAINLINK_HOME/brainlink.config.json`, `--no-migrate` to skip migration, and `--no-index` to skip post-migration indexing.
 `config doctor` is dry-run by default; use `--fix` to apply safe config normalization and allowlist fixes.
 
@@ -667,6 +713,18 @@ blink migrate-vault --from ~/.brainlink/vault --to ./team-vault --report ./migra
 Runs explicit markdown migration between vaults while preserving conflicts as `.conflict-<timestamp>` files.  
 Use `--dry-run` to preview `copied`, `conflicted` and `unchanged` counts before writing.
 
+### `db-import`
+
+```bash
+blink db-import --vault ./team-vault
+blink db-import --vault ./team-vault --db ./legacy/brainlink.db
+blink db-import --vault ./team-vault --db ./legacy/brainlink.db --table legacy_notes --dry-run
+```
+
+Imports durable memory from a legacy SQLite database into Markdown notes (`agents/<agent-id>/*.md`) and reindexes by default.
+When `--db` is omitted, Brainlink auto-detects common legacy paths such as `<vault>/.brainlink/brainlink.db`.
+Use `--agent <id>` to force all imported rows into one namespace, `--limit` for incremental imports, `--dry-run` to preview without writing files, and `--no-index` to defer reindexing.
+
 ### `init`
 
 ```bash
@@ -691,6 +749,28 @@ blink add "Note Title" --vault ./vault --content-file ./notes.md --no-auto-index
 
 Creates a Markdown note under `agents/<agent-id>/`. Common secret patterns are blocked by default; use `--allow-sensitive` only for an intentionally protected vault.
 To avoid disconnected memory, Brainlink auto-adds a fallback wiki edge when a note is written without links, creating agent hub notes when needed.
+`add` also returns `possibleDuplicates` (exact hash + semantic candidates) so agents can resolve duplicate memory right after writes.
+
+### `dedupe`
+
+```bash
+blink dedupe --vault ./vault --json
+blink dedupe --vault ./vault --agent coding-agent --limit 20 --min-score 0.92 --json
+blink dedupe --vault ./vault --no-semantic --json
+```
+
+Detects `possibleDuplicate` pairs using exact content hashes and optional semantic similarity.
+
+### `dedupe-resolve`
+
+```bash
+blink dedupe-resolve --vault ./vault --left agents/shared/a.md --right agents/shared/b.md --action merge --json
+blink dedupe-resolve --vault ./vault --left agents/shared/a.md --right agents/shared/b.md --action link --json
+blink dedupe-resolve --vault ./vault --left agents/shared/a.md --right agents/shared/b.md --action ignore --json
+```
+
+Resolves a duplicate pair with `merge`, `link` or `ignore`.
+When action is not `merge`, Brainlink still creates a low-priority related edge (`#related-to`) so notes remain connected.
 
 ### `index`
 
@@ -701,6 +781,38 @@ blink index --vault ./vault
 
 Rebuilds the local index from Markdown files.
 
+### `bench`
+
+```bash
+blink bench --vault ./vault
+blink bench --vault ./vault --watch
+blink bench --vault ./vault --watch --debounce 500
+blink bench --vault ./vault --json
+```
+
+Runs indexing with realtime phase telemetry (`start`, `scan`, `parse`, `embed`, `persist`, `packs`, `complete`) and prints a benchmark summary at the end of each run.
+
+Summary includes compression behavior for `.blpk` packs when rebuild happens:
+- pack rebuild reason
+- pack count and pack build duration
+- uncompressed input bytes vs compressed output bytes
+- saved percentage
+- objective guardrails (minimum savings and maximum latency regression thresholds)
+
+Use `--watch` to keep benchmarking incremental reindex runs after Markdown changes (local filesystem vaults only).
+When `.brainlink/search-packs/manifest.json` is missing but `.blpk` files exist, Brainlink repairs the manifest first and avoids unnecessary full pack rebuild on small edits.
+
+### `pack-backup`
+
+```bash
+blink pack-backup --vault ./vault
+blink pack-backup --vault ./vault --output ./vault/.brainlink/backups/custom.blpkbak.gz
+blink pack-backup --vault ./vault --json
+```
+
+Creates an offline backup artifact of encrypted search packs with a second compression pass.
+This is intentionally outside the online retrieval path (`index`, `search`, `context`).
+
 ### `agents`
 
 ```bash
@@ -728,6 +840,7 @@ Modes:
 - `semantic`: local deterministic embedding similarity only.
 
 Hybrid results are cached in-memory for a short TTL and invalidated automatically when the local index file changes.
+Context selection uses a middle-out strategy: it starts from the strongest chunk in a note and expands to neighboring chunks while respecting token budget.
 
 ### `context`
 
@@ -738,6 +851,7 @@ blink context "question" --vault ./vault --agent coding-agent --mode hybrid --js
 ```
 
 Builds a compact context package for an agent.
+Repeated calls with the same vault, agent, query, mode and token/limit settings are served from a short in-memory cache while the index is unchanged.
 
 ### `links`
 
@@ -822,9 +936,15 @@ Watches Markdown files and rebuilds the index when notes change.
 ```bash
 blink server --watch
 blink server --vault ./vault --watch
+blink server --vault ./vault --watch --no-open
 ```
 
 Starts the local read-only graph UI and HTTP API.
+By default, it tries to open a native desktop GUI window for the graph URL.
+On Linux, native GUI is disabled by default; enable it with `BRAINLINK_LINUX_NATIVE_GUI=1`.
+If native GUI launch is unavailable, it falls back to dedicated app-window mode and then browser open.
+When fallback opens Chromium-family browsers on Linux, Brainlink automatically uses compatibility launch flags for stable rendering on Ubuntu/Wayland setups.
+Use `--no-open` to skip that behavior.
 
 The HTTP server only binds to loopback hosts such as `127.0.0.1`, `localhost` or `::1`.
 
@@ -865,6 +985,13 @@ If no `vault` is configured and no `--vault` flag is passed, Brainlink uses `$HO
   "embeddingProvider": "local",
   "defaultSearchMode": "hybrid",
   "chunkSize": 1200,
+  "searchPack": {
+    "rowChunkSize": 5000,
+    "compressionLevel": 5,
+    "useDictionary": true,
+    "guardrailMinSavingsPercent": 8,
+    "guardrailMaxLatencyRegressionPercent": 5
+  },
   "agentProfiles": {
     "coding-agent": {
       "defaultSearchMode": "semantic",
@@ -1025,6 +1152,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md).
 ## License
 
 MIT. See [LICENSE](LICENSE).
+Copyright (c) 2026 Substructa. See [COPYRIGHT.md](COPYRIGHT.md).
 
 ### Memory Optimization Loop (1-7)
 

package/dist/application/auto-migrate-configured-vault.js

@@ -0,0 +1,37 @@
+import { indexVault } from './index-vault.js';
+import { migrateVaultContent } from './migrate-vault.js';
+import { getLastConfiguredVaultForKey, setLastConfiguredVaultForKey } from '../infrastructure/vault-migration-state.js';
+export const autoMigrateConfiguredVaultIfChanged = async (input) => {
+    const configKey = input.configKey.trim();
+    const configuredVault = input.configuredVault.trim();
+    if (configKey.length === 0 || configuredVault.length === 0) {
+        return {
+            changed: false,
+            migrated: false
+        };
+    }
+    const previousVault = await getLastConfiguredVaultForKey(configKey);
+    if (!previousVault) {
+        await setLastConfiguredVaultForKey(configKey, configuredVault);
+        return {
+            changed: false,
+            migrated: false
+        };
+    }
+    if (previousVault === configuredVault) {
+        return {
+            changed: false,
+            migrated: false
+        };
+    }
+    const migration = await migrateVaultContent(previousVault, configuredVault);
+    const shouldIndex = migration.copied + migration.conflicted > 0;
+    if (shouldIndex) {
+        await indexVault(configuredVault);
+    }
+    await setLastConfiguredVaultForKey(configKey, configuredVault);
+    return {
+        changed: true,
+        migrated: shouldIndex
+    };
+};

package/dist/application/build-context.js

@@ -1,13 +1,74 @@
+import { stat } from 'node:fs/promises';
 import { formatContextPackage, selectContextSections } from '../domain/context.js';
+import { indexStoragePath } from '../infrastructure/file-index.js';
+import { searchVolatileMemory, volatileMemoryStoragePath } from '../infrastructure/volatile-memory.js';
 import { searchKnowledge } from './search-knowledge.js';
+const contextCacheTtlMs = 45_000;
+const contextCacheMaxEntries = 200;
+const contextCache = new Map();
+const readFileSignature = async (path) => {
+    try {
+        const info = await stat(path);
+        return `${Math.floor(info.mtimeMs)}:${info.size}`;
+    }
+    catch {
+        return '0:0';
+    }
+};
+const readContextDataSignature = async (vaultPath) => `${await readFileSignature(indexStoragePath(vaultPath))}|${await readFileSignature(volatileMemoryStoragePath(vaultPath))}`;
+const toCacheKey = (vaultPath, query, limit, maxTokens, agentId, mode) => JSON.stringify({
+    vaultPath,
+    query: query.trim().toLowerCase(),
+    limit,
+    maxTokens,
+    agentId: agentId?.trim().toLowerCase() ?? '*',
+    mode: mode ?? 'default'
+});
+const contextCacheGet = (key, dataSignature) => {
+    const entry = contextCache.get(key);
+    if (!entry) {
+        return undefined;
+    }
+    const fresh = Date.now() - entry.createdAt <= contextCacheTtlMs && entry.dataSignature === dataSignature;
+    if (!fresh) {
+        contextCache.delete(key);
+        return undefined;
+    }
+    return entry.context;
+};
+const contextCacheSet = (entry) => {
+    contextCache.set(entry.key, entry);
+    if (contextCache.size <= contextCacheMaxEntries) {
+        return;
+    }
+    const overflow = contextCache.size - contextCacheMaxEntries;
+    const keys = Array.from(contextCache.keys()).slice(0, overflow);
+    keys.forEach((key) => contextCache.delete(key));
+};
 export const buildContextPackage = async (vaultPath, query, limit, maxTokens, agentId, mode) => {
+    const cacheKey = toCacheKey(vaultPath, query, limit, maxTokens, agentId, mode);
+    const dataSignature = await readContextDataSignature(vaultPath);
+    const cached = contextCacheGet(cacheKey, dataSignature);
+    if (cached) {
+        return cached;
+    }
     const results = await searchKnowledge(vaultPath, query, limit, agentId, mode);
-    const sections = selectContextSections(results, maxTokens);
-    return {
+    const durableSections = selectContextSections(results, maxTokens);
+    const volatileSections = await searchVolatileMemory(vaultPath, query, Math.min(3, limit), agentId, mode ?? 'hybrid');
+    const sections = [...volatileSections, ...durableSections];
+    const context = {
         query,
         sections,
-        content: formatContextPackage(query, sections)
+        content: formatContextPackage(query, sections),
+        ...(volatileSections.length > 0 ? { volatileSections } : {})
     };
+    contextCacheSet({
+        key: cacheKey,
+        createdAt: Date.now(),
+        dataSignature,
+        context
+    });
+    return context;
 };
 export const buildContext = async (vaultPath, query, limit, maxTokens, agentId, mode) => {
     const contextPackage = await buildContextPackage(vaultPath, query, limit, maxTokens, agentId, mode);

package/dist/application/dedupe-notes.js

@@ -0,0 +1,226 @@
+import { createHash } from 'node:crypto';
+import { createEmbeddingBuckets, createLocalEmbedding, cosineSimilarity } from '../domain/embeddings.js';
+import { parseMarkdownDocument } from '../domain/markdown.js';
+import { writeMarkdownFile, ensureVault, readMarkdownFiles } from '../infrastructure/file-system-vault.js';
+import { indexVault } from './index-vault.js';
+const tokenPattern = /[\p{L}\p{N}_-]+/gu;
+const frontmatterPattern = /^---\n[\s\S]*?\n---\n?/m;
+const rootHeadingPattern = /^#\s+.+\n+/m;
+const maxCandidatesPerBucket = 240;
+const normalizePath = (path) => path.replaceAll('\\', '/').replace(/^\.\//, '');
+const toComparableBody = (content) => content
+    .replace(frontmatterPattern, '')
+    .replace(rootHeadingPattern, '')
+    .replaceAll('\r\n', '\n')
+    .trim();
+const normalizeStrictContent = (content) => toComparableBody(content);
+const normalizeSemanticContent = (content) => toComparableBody(content)
+    .replace(/\s+/g, ' ')
+    .trim();
+const toHash = (value) => createHash('sha256').update(value, 'utf8').digest('hex');
+const toCandidateId = (leftPath, rightPath) => [normalizePath(leftPath), normalizePath(rightPath)].sort((left, right) => left.localeCompare(right)).join('|');
+const hasSharedTokens = (left, right) => {
+    const leftTokens = new Set((left.match(tokenPattern) ?? []).map((token) => token.toLowerCase()).filter((token) => token.length > 2));
+    const rightTokens = new Set((right.match(tokenPattern) ?? []).map((token) => token.toLowerCase()).filter((token) => token.length > 2));
+    if (leftTokens.size === 0 || rightTokens.size === 0) {
+        return false;
+    }
+    for (const token of leftTokens) {
+        if (rightTokens.has(token)) {
+            return true;
+        }
+    }
+    return false;
+};
+const relatedMarker = (targetTitle) => `Related: [[${targetTitle}]] priority: low #related-to`;
+const ensureRelatedEdgeLine = (content, targetTitle) => {
+    const linkPattern = new RegExp(`\\[\\[\\s*${targetTitle.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*(?:[\\]|#])?`, 'i');
+    if (linkPattern.test(content)) {
+        return content;
+    }
+    const trimmed = content.trimEnd();
+    return `${trimmed}\n\n${relatedMarker(targetTitle)}\n`;
+};
+const ensureMergedMarker = (content, targetTitle) => {
+    const marker = `Merged into [[${targetTitle}]]`;
+    if (content.includes(marker)) {
+        return content;
+    }
+    return `${content.trimEnd()}\n\n${marker} priority: low #related-to\n`;
+};
+const appendMergedContent = (baseContent, mergedTitle, mergedContent) => {
+    const marker = `## Merged Memory From [[${mergedTitle}]]`;
+    if (baseContent.includes(marker)) {
+        return baseContent;
+    }
+    const mergedBody = normalizeSemanticContent(mergedContent);
+    return `${baseContent.trimEnd()}\n\n${marker}\n\n${mergedBody}\n`;
+};
+const loadNoteRecords = async (vaultPath, agentId) => {
+    const absoluteVaultPath = await ensureVault(vaultPath);
+    const files = await readMarkdownFiles(vaultPath);
+    return files
+        .map((file) => {
+        const parsed = parseMarkdownDocument({
+            absolutePath: file.absolutePath,
+            vaultPath: absoluteVaultPath,
+            content: file.content,
+            createdAt: file.createdAt,
+            updatedAt: file.updatedAt
+        });
+        const strict = normalizeStrictContent(parsed.content);
+        const semantic = normalizeSemanticContent(parsed.content);
+        const embedding = createLocalEmbedding(`${parsed.title}\n${semantic}`);
+        return {
+            title: parsed.title,
+            path: normalizePath(parsed.path),
+            agentId: parsed.agentId,
+            content: parsed.content,
+            normalizedStrictContent: strict,
+            semanticContent: semantic,
+            embedding,
+            buckets: createEmbeddingBuckets(embedding, 20)
+        };
+    })
+        .filter((record) => (agentId ? record.agentId === agentId : true));
+};
+const pairToCandidate = (left, right, kind, score, reason) => ({
+    id: toCandidateId(left.path, right.path),
+    possibleDuplicate: true,
+    kind,
+    score: Number(score.toFixed(4)),
+    left: {
+        title: left.title,
+        path: left.path,
+        agentId: left.agentId
+    },
+    right: {
+        title: right.title,
+        path: right.path,
+        agentId: right.agentId
+    },
+    reason
+});
+const indexCandidatePairs = (notes) => {
+    const bucketMap = new Map();
+    notes.forEach((note, index) => {
+        note.buckets.forEach((bucket) => {
+            const current = bucketMap.get(bucket) ?? [];
+            if (current.length < maxCandidatesPerBucket) {
+                current.push(index);
+                bucketMap.set(bucket, current);
+            }
+        });
+    });
+    const pairKeys = new Set();
+    const pairs = [];
+    bucketMap.forEach((indexes) => {
+        for (let leftIndex = 0; leftIndex < indexes.length; leftIndex += 1) {
+            for (let rightIndex = leftIndex + 1; rightIndex < indexes.length; rightIndex += 1) {
+                const left = Math.min(indexes[leftIndex] ?? 0, indexes[rightIndex] ?? 0);
+                const right = Math.max(indexes[leftIndex] ?? 0, indexes[rightIndex] ?? 0);
+                const key = `${left}|${right}`;
+                if (!pairKeys.has(key)) {
+                    pairKeys.add(key);
+                    pairs.push([left, right]);
+                }
+            }
+        }
+    });
+    return pairs;
+};
+export const scanDuplicateNotes = async (vaultPath, options = {}) => {
+    const notes = await loadNoteRecords(vaultPath, options.agentId);
+    if (notes.length < 2) {
+        return [];
+    }
+    const minSemanticScore = options.minSemanticScore ?? 0.92;
+    const includeSemantic = options.includeSemantic !== false;
+    const seen = new Map();
+    const byHash = notes.reduce((state, note) => {
+        const key = toHash(note.normalizedStrictContent);
+        const current = state.get(key) ?? [];
+        current.push(note);
+        state.set(key, current);
+        return state;
+    }, new Map());
+    byHash.forEach((group) => {
+        if (group.length < 2) {
+            return;
+        }
+        const [base, ...rest] = group.sort((left, right) => left.path.localeCompare(right.path));
+        rest.forEach((note) => {
+            const candidate = pairToCandidate(base, note, 'exact', 1, 'Exact content hash match');
+            seen.set(candidate.id, candidate);
+        });
+    });
+    if (includeSemantic) {
+        const pairs = indexCandidatePairs(notes);
+        pairs.forEach(([leftIndex, rightIndex]) => {
+            const left = notes[leftIndex];
+            const right = notes[rightIndex];
+            if (!left || !right || left.path === right.path) {
+                return;
+            }
+            const id = toCandidateId(left.path, right.path);
+            if (seen.has(id)) {
+                return;
+            }
+            const score = cosineSimilarity(left.embedding, right.embedding);
+            const titleShared = hasSharedTokens(left.title, right.title);
+            const contentShared = hasSharedTokens(left.semanticContent, right.semanticContent);
+            if (score >= minSemanticScore && (titleShared || contentShared || score >= 0.975)) {
+                const candidate = pairToCandidate(left, right, 'semantic', score, 'High semantic similarity');
+                seen.set(id, candidate);
+            }
+        });
+    }
+    const focusPath = options.focusPath ? normalizePath(options.focusPath) : undefined;
+    const limited = Array.from(seen.values())
+        .filter((item) => (focusPath ? item.left.path === focusPath || item.right.path === focusPath : true))
+        .sort((left, right) => right.score - left.score || left.left.path.localeCompare(right.left.path))
+        .slice(0, Math.max(1, options.limit ?? 25));
+    return limited;
+};
+export const resolveDuplicateNotes = async (vaultPath, options) => {
+    const leftPath = normalizePath(options.leftPath);
+    const rightPath = normalizePath(options.rightPath);
+    if (leftPath === rightPath) {
+        throw new Error('leftPath and rightPath must be different notes.');
+    }
+    const notes = await loadNoteRecords(vaultPath);
+    const byPath = new Map(notes.map((note) => [note.path, note]));
+    const left = byPath.get(leftPath);
+    const right = byPath.get(rightPath);
+    if (!left || !right) {
+        throw new Error(`Duplicate resolution paths were not found in vault index source: ${leftPath}, ${rightPath}`);
+    }
+    const updates = new Map();
+    const leftRelated = ensureRelatedEdgeLine(left.content, right.title);
+    const rightRelated = ensureRelatedEdgeLine(right.content, left.title);
+    if (options.action === 'link') {
+        updates.set(left.path, leftRelated);
+        updates.set(right.path, rightRelated);
+    }
+    else if (options.action === 'ignore') {
+        updates.set(left.path, leftRelated);
+    }
+    else {
+        const mergedLeft = appendMergedContent(leftRelated, right.title, right.content);
+        const mergedRight = ensureMergedMarker(rightRelated, left.title);
+        updates.set(left.path, mergedLeft);
+        updates.set(right.path, mergedRight);
+    }
+    for (const [path, content] of updates) {
+        await writeMarkdownFile(vaultPath, path, content);
+    }
+    const shouldIndex = options.autoIndex !== false;
+    const index = shouldIndex ? await indexVault(vaultPath) : undefined;
+    return {
+        action: options.action,
+        leftPath,
+        rightPath,
+        updatedPaths: Array.from(updates.keys()).sort((leftValue, rightValue) => leftValue.localeCompare(rightValue)),
+        ...(index ? { index } : {})
+    };
+};