@swarmvaultai/engine 0.1.2 → 0.1.3

package/README.md

@@ -2,7 +2,7 @@
 
 `@swarmvaultai/engine` is the runtime library behind SwarmVault.
 
-It provides the ingest, compile, query, lint, provider, graph, search, and viewer-serving primitives used by the CLI.
+It exposes the primitives for initializing a workspace, ingesting sources, importing an inbox, compiling a wiki, querying the vault, running lint, serving the graph viewer, watching the inbox, and exposing the vault over MCP.
 
 ## Who This Is For
 
@@ -10,7 +10,8 @@ Use this package if you want to:
 
 - build your own interface on top of the SwarmVault runtime
 - integrate vault operations into another Node application
-- customize provider loading or workspace orchestration without shelling out to the CLI
+- embed watch or MCP behavior without shelling out to the CLI
+- customize provider loading or orchestration in code
 
 If you only want to use SwarmVault as a tool, install `@swarmvaultai/cli` instead.
 
@@ -19,48 +20,54 @@ If you only want to use SwarmVault as a tool, install `@swarmvaultai/cli` instea
 ```ts
 import {
   compileVault,
-  createProvider,
+  createMcpServer,
   defaultVaultConfig,
+  importInbox,
   ingestInput,
   initVault,
   installAgent,
   lintVault,
   loadVaultConfig,
   queryVault,
-  startGraphServer
+  searchVault,
+  startGraphServer,
+  startMcpServer,
+  watchVault,
 } from "@swarmvaultai/engine";
 ```
 
-The engine also exports the main runtime types for providers, graph artifacts, pages, manifests, and lint findings.
+The engine also exports the main runtime types for providers, graph artifacts, pages, manifests, query results, lint findings, and watch records.
 
 ## Example
 
 ```ts
-import { compileVault, ingestInput, initVault, queryVault } from "@swarmvaultai/engine";
+import { compileVault, importInbox, initVault, queryVault, watchVault } from "@swarmvaultai/engine";
 
 const rootDir = process.cwd();
 
 await initVault(rootDir);
-await ingestInput(rootDir, "./research.md");
+await importInbox(rootDir);
 await compileVault(rootDir);
 
 const result = await queryVault(rootDir, "What changed most recently?", true);
 console.log(result.answer);
+
+const watcher = await watchVault(rootDir, { lint: true });
 ```
 
 ## Provider Model
 
 The engine supports:
 
+- `heuristic`
 - `openai`
-- `ollama`
 - `anthropic`
 - `gemini`
+- `ollama`
 - `openai-compatible`
 - `custom`
-- `heuristic`
 
-Providers are validated through capabilities such as:
+Providers are capability-driven. Each provider declares support for features such as:
 
 - `chat`
 - `structured`
@@ -70,29 +77,56 @@ Providers are validated through capabilities such as:
 - `streaming`
 - `local`
 
-This matters because many "OpenAI-compatible" providers implement only part of the OpenAI surface. The engine is designed to route by capability, not by brand name.
+This matters because many "OpenAI-compatible" backends only implement part of the OpenAI surface.
 
-## Artifacts
+## Main Engine Surfaces
 
-Running the engine produces a local workspace with four main areas:
+### Ingest
 
-- `raw/`: immutable source inputs and captured assets
-- `wiki/`: generated markdown pages and saved outputs
-- `state/`: manifests, extracts, graph state, compile state, and search index
-- `agent/`: generated agent rules
+- `ingestInput(rootDir, input)` ingests a local path or URL
+- `importInbox(rootDir, inputDir?)` recursively imports supported inbox files and browser-clipper style bundles
+
+### Compile + Query
+
+- `compileVault(rootDir)` writes wiki pages, graph data, and search state
+- `queryVault(rootDir, question, save)` answers against the compiled vault
+- `searchVault(rootDir, query, limit)` searches compiled pages directly
+
+### Automation
 
-Important artifacts include:
+- `watchVault(rootDir, options)` watches the inbox and appends run records to `state/jobs.ndjson`
+- `lintVault(rootDir)` runs health and anti-drift checks
 
-- `state/graph.json`
-- `state/search.sqlite`
-- `wiki/index.md`
-- `wiki/sources/*.md`
-- `wiki/concepts/*.md`
-- `wiki/entities/*.md`
-- `wiki/outputs/*.md`
+### MCP
+
+- `createMcpServer(rootDir)` creates an MCP server instance
+- `startMcpServer(rootDir)` runs the MCP server over stdio
+
+The MCP surface includes tools for workspace info, page search, page reads, source listing, querying, ingestion, compile, and lint, along with resources for config, graph, manifests, and page content.
+
+## Artifacts
+
+Running the engine produces a local workspace with these main areas:
+
+- `inbox/`: capture staging area for markdown bundles and imported files
+- `raw/sources/`: immutable source copies
+- `raw/assets/`: copied attachments referenced by ingested markdown bundles
+- `wiki/`: generated markdown pages and saved outputs
+- `state/manifests/`: source manifests
+- `state/extracts/`: extracted text
+- `state/analyses/`: model analysis output
+- `state/graph.json`: compiled graph
+- `state/search.sqlite`: full-text index
+- `state/jobs.ndjson`: watch-mode automation logs
 
 ## Notes
 
 - The engine expects Node `>=24`
 - The local search layer currently uses the built-in `node:sqlite` module, which may emit an experimental warning in Node 24
 - The viewer source lives in the companion `@swarmvaultai/viewer` package, and the built assets are bundled into the engine package for CLI installs
+
+## Links
+
+- Website: https://www.swarmvault.ai
+- Docs: https://www.swarmvault.ai/docs
+- GitHub: https://github.com/swarmclawai/swarmvault

package/dist/index.d.ts

@@ -1,4 +1,6 @@
 import { z } from 'zod';
+import { Readable, Writable } from 'node:stream';
+import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
 
 declare const providerCapabilitySchema: z.ZodEnum<{
     responses: "responses";
@@ -66,6 +68,7 @@ interface VaultConfig {
         wikiDir: string;
         stateDir: string;
         agentDir: string;
+        inboxDir: string;
     };
     providers: Record<string, ProviderConfig>;
     tasks: {
@@ -82,9 +85,12 @@ interface VaultConfig {
 interface ResolvedPaths {
     rootDir: string;
     rawDir: string;
+    rawSourcesDir: string;
+    rawAssetsDir: string;
     wikiDir: string;
     stateDir: string;
     agentDir: string;
+    inboxDir: string;
     manifestsDir: string;
     extractsDir: string;
     analysesDir: string;
@@ -92,8 +98,14 @@ interface ResolvedPaths {
     graphPath: string;
     searchDbPath: string;
     compileStatePath: string;
+    jobsLogPath: string;
     configPath: string;
 }
+interface SourceAttachment {
+    path: string;
+    mimeType: string;
+    originalPath?: string;
+}
 interface SourceManifest {
     sourceId: string;
     title: string;
@@ -107,6 +119,7 @@ interface SourceManifest {
     contentHash: string;
     createdAt: string;
     updatedAt: string;
+    attachments?: SourceAttachment[];
 }
 interface AnalyzedTerm {
     id: string;
@@ -193,6 +206,38 @@ interface LintFinding {
     message: string;
     pagePath?: string;
 }
+interface InboxImportSkip {
+    path: string;
+    reason: string;
+}
+interface InboxImportResult {
+    inputDir: string;
+    scannedCount: number;
+    attachmentCount: number;
+    imported: SourceManifest[];
+    skipped: InboxImportSkip[];
+}
+interface WatchOptions {
+    lint?: boolean;
+    debounceMs?: number;
+}
+interface WatchRunRecord {
+    startedAt: string;
+    finishedAt: string;
+    durationMs: number;
+    inputDir: string;
+    reasons: string[];
+    importedCount: number;
+    scannedCount: number;
+    attachmentCount: number;
+    changedPages: string[];
+    lintFindingCount?: number;
+    success: boolean;
+    error?: string;
+}
+interface WatchController {
+    close(): Promise<void>;
+}
 
 declare function defaultVaultConfig(): VaultConfig;
 declare function resolvePaths(rootDir: string, config?: VaultConfig, configPath?: string): ResolvedPaths;
@@ -206,12 +251,32 @@ declare function initWorkspace(rootDir: string): Promise<{
 }>;
 
 declare function ingestInput(rootDir: string, input: string): Promise<SourceManifest>;
+declare function importInbox(rootDir: string, inputDir?: string): Promise<InboxImportResult>;
 declare function listManifests(rootDir: string): Promise<SourceManifest[]>;
 declare function readExtractedText(rootDir: string, manifest: SourceManifest): Promise<string | undefined>;
 
 declare function initVault(rootDir: string): Promise<void>;
 declare function compileVault(rootDir: string): Promise<CompileResult>;
 declare function queryVault(rootDir: string, question: string, save?: boolean): Promise<QueryResult>;
+declare function searchVault(rootDir: string, query: string, limit?: number): Promise<SearchResult[]>;
+declare function listPages(rootDir: string): Promise<GraphPage[]>;
+declare function readPage(rootDir: string, relativePath: string): Promise<{
+    path: string;
+    title: string;
+    frontmatter: Record<string, unknown>;
+    content: string;
+} | null>;
+declare function getWorkspaceInfo(rootDir: string): Promise<{
+    rootDir: string;
+    configPath: string;
+    rawDir: string;
+    wikiDir: string;
+    stateDir: string;
+    agentDir: string;
+    inboxDir: string;
+    sourceCount: number;
+    pageCount: number;
+}>;
 declare function lintVault(rootDir: string): Promise<LintFinding[]>;
 declare function bootstrapDemo(rootDir: string, input?: string): Promise<{
     manifestId?: string;
@@ -226,8 +291,15 @@ declare function startGraphServer(rootDir: string, port?: number): Promise<{
     close: () => Promise<void>;
 }>;
 
+declare function createMcpServer(rootDir: string): Promise<McpServer>;
+declare function startMcpServer(rootDir: string, stdin?: Readable, stdout?: Writable): Promise<{
+    close: () => Promise<void>;
+}>;
+
+declare function watchVault(rootDir: string, options?: WatchOptions): Promise<WatchController>;
+
 declare function createProvider(id: string, config: ProviderConfig, rootDir: string): Promise<ProviderAdapter>;
 declare function getProviderForTask(rootDir: string, task: keyof Awaited<ReturnType<typeof loadVaultConfig>>["config"]["tasks"]): Promise<ProviderAdapter>;
 declare function assertProviderCapability(provider: ProviderAdapter, capability: ProviderCapability): void;
 
-export { type AnalyzedTerm, type ClaimStatus, type CompileResult, type Freshness, type GenerationAttachment, type GenerationRequest, type GenerationResponse, type GraphArtifact, type GraphEdge, type GraphNode, type GraphPage, type LintFinding, type PageKind, type Polarity, type ProviderAdapter, type ProviderCapability, type ProviderConfig, type ProviderType, type QueryResult, type ResolvedPaths, type SearchResult, type SourceAnalysis, type SourceClaim, type SourceManifest, type VaultConfig, assertProviderCapability, bootstrapDemo, compileVault, createProvider, defaultVaultConfig, getProviderForTask, ingestInput, initVault, initWorkspace, installAgent, installConfiguredAgents, lintVault, listManifests, loadVaultConfig, providerCapabilitySchema, providerTypeSchema, queryVault, readExtractedText, resolvePaths, startGraphServer };
+export { type AnalyzedTerm, type ClaimStatus, type CompileResult, type Freshness, type GenerationAttachment, type GenerationRequest, type GenerationResponse, type GraphArtifact, type GraphEdge, type GraphNode, type GraphPage, type InboxImportResult, type InboxImportSkip, type LintFinding, type PageKind, type Polarity, type ProviderAdapter, type ProviderCapability, type ProviderConfig, type ProviderType, type QueryResult, type ResolvedPaths, type SearchResult, type SourceAnalysis, type SourceAttachment, type SourceClaim, type SourceManifest, type VaultConfig, type WatchController, type WatchOptions, type WatchRunRecord, assertProviderCapability, bootstrapDemo, compileVault, createMcpServer, createProvider, defaultVaultConfig, getProviderForTask, getWorkspaceInfo, importInbox, ingestInput, initVault, initWorkspace, installAgent, installConfiguredAgents, lintVault, listManifests, listPages, loadVaultConfig, providerCapabilitySchema, providerTypeSchema, queryVault, readExtractedText, readPage, resolvePaths, searchVault, startGraphServer, startMcpServer, watchVault };

package/dist/index.js

@@ -36,6 +36,11 @@ async function writeJsonFile(filePath, value) {
   await fs.writeFile(filePath, `${JSON.stringify(value, null, 2)}
 `, "utf8");
 }
+async function appendJsonLine(filePath, value) {
+  await ensureDir(path.dirname(filePath));
+  await fs.appendFile(filePath, `${JSON.stringify(value)}
+`, "utf8");
+}
 async function writeFileIfChanged(filePath, content) {
   await ensureDir(path.dirname(filePath));
   if (await fileExists(filePath)) {
@@ -88,6 +93,21 @@ function truncate(value, maxLength) {
   }
   return `${value.slice(0, maxLength - 3)}...`;
 }
+async function listFilesRecursive(rootDir) {
+  const entries = await fs.readdir(rootDir, { withFileTypes: true }).catch(() => []);
+  const files = [];
+  for (const entry of entries) {
+    const absolutePath = path.join(rootDir, entry.name);
+    if (entry.isDirectory()) {
+      files.push(...await listFilesRecursive(absolutePath));
+      continue;
+    }
+    if (entry.isFile()) {
+      files.push(absolutePath);
+    }
+  }
+  return files;
+}
 
 // src/types.ts
 import { z } from "zod";
@@ -130,7 +150,8 @@ var vaultConfigSchema = z2.object({
     rawDir: z2.string().min(1),
     wikiDir: z2.string().min(1),
     stateDir: z2.string().min(1),
-    agentDir: z2.string().min(1)
+    agentDir: z2.string().min(1),
+    inboxDir: z2.string().min(1)
   }),
   providers: z2.record(z2.string(), providerConfigSchema),
   tasks: z2.object({
@@ -150,7 +171,8 @@ function defaultVaultConfig() {
       rawDir: "raw",
       wikiDir: "wiki",
       stateDir: "state",
-      agentDir: "agent"
+      agentDir: "agent",
+      inboxDir: "inbox"
     },
     providers: {
       local: {
@@ -185,15 +207,21 @@ async function findConfigPath(rootDir) {
 function resolvePaths(rootDir, config, configPath = path2.join(rootDir, PRIMARY_CONFIG_FILENAME)) {
   const effective = config ?? defaultVaultConfig();
   const rawDir = path2.resolve(rootDir, effective.workspace.rawDir);
+  const rawSourcesDir = path2.join(rawDir, "sources");
+  const rawAssetsDir = path2.join(rawDir, "assets");
   const wikiDir = path2.resolve(rootDir, effective.workspace.wikiDir);
   const stateDir = path2.resolve(rootDir, effective.workspace.stateDir);
   const agentDir = path2.resolve(rootDir, effective.workspace.agentDir);
+  const inboxDir = path2.resolve(rootDir, effective.workspace.inboxDir);
   return {
     rootDir,
     rawDir,
+    rawSourcesDir,
+    rawAssetsDir,
     wikiDir,
     stateDir,
     agentDir,
+    inboxDir,
     manifestsDir: path2.join(stateDir, "manifests"),
     extractsDir: path2.join(stateDir, "extracts"),
     analysesDir: path2.join(stateDir, "analyses"),
@@ -201,6 +229,7 @@ function resolvePaths(rootDir, config, configPath = path2.join(rootDir, PRIMARY_
     graphPath: path2.join(stateDir, "graph.json"),
     searchDbPath: path2.join(stateDir, "search.sqlite"),
     compileStatePath: path2.join(stateDir, "compile-state.json"),
+    jobsLogPath: path2.join(stateDir, "jobs.ndjson"),
     configPath
   };
 }
@@ -222,9 +251,12 @@ async function initWorkspace(rootDir) {
     ensureDir(paths.wikiDir),
     ensureDir(paths.stateDir),
     ensureDir(paths.agentDir),
+    ensureDir(paths.inboxDir),
     ensureDir(paths.manifestsDir),
     ensureDir(paths.extractsDir),
-    ensureDir(paths.analysesDir)
+    ensureDir(paths.analysesDir),
+    ensureDir(paths.rawSourcesDir),
+    ensureDir(paths.rawAssetsDir)
   ]);
   if (!await fileExists(configPath)) {
     await writeJsonFile(configPath, config);
@@ -253,6 +285,10 @@ async function appendLogEntry(rootDir, action, title, lines = []) {
   await fs2.writeFile(logPath, `${existing}${entry}
 `, "utf8");
 }
+async function appendWatchRun(rootDir, run) {
+  const { paths } = await initWorkspace(rootDir);
+  await appendJsonLine(paths.jobsLogPath, run);
+}
 
 // src/ingest.ts
 function inferKind(mimeType, filePath) {
@@ -280,6 +316,50 @@ function titleFromText(fallback, content) {
 function guessMimeType(target) {
   return mime.lookup(target) || "application/octet-stream";
 }
+function buildCompositeHash(payloadBytes, attachments = []) {
+  if (!attachments.length) {
+    return sha256(payloadBytes);
+  }
+  const attachmentSignature = attachments.map((attachment) => `${attachment.relativePath}:${sha256(attachment.bytes)}`).sort().join("|");
+  return sha256(`${sha256(payloadBytes)}|${attachmentSignature}`);
+}
+function sanitizeAssetRelativePath(value) {
+  const normalized = path4.posix.normalize(value.replace(/\\/g, "/"));
+  const segments = normalized.split("/").filter(Boolean).map((segment) => {
+    if (segment === ".") {
+      return "";
+    }
+    if (segment === "..") {
+      return "_up";
+    }
+    return segment;
+  }).filter(Boolean);
+  return segments.join("/") || "asset";
+}
+function normalizeLocalReference(value) {
+  const trimmed = value.trim().replace(/^<|>$/g, "");
+  const [withoutTitle] = trimmed.split(/\s+(?=(?:[^"]*"[^"]*")*[^"]*$)/, 1);
+  const candidate = withoutTitle.split("#")[0]?.split("?")[0]?.trim();
+  if (!candidate) {
+    return null;
+  }
+  const lowered = candidate.toLowerCase();
+  if (lowered.startsWith("http://") || lowered.startsWith("https://") || lowered.startsWith("data:") || lowered.startsWith("mailto:") || lowered.startsWith("#") || path4.isAbsolute(candidate)) {
+    return null;
+  }
+  return candidate.replace(/\\/g, "/");
+}
+function extractMarkdownReferences(content) {
+  const references = [];
+  const linkPattern = /!?\[[^\]]*]\(([^)]+)\)/g;
+  for (const match of content.matchAll(linkPattern)) {
+    const normalized = normalizeLocalReference(match[1] ?? "");
+    if (normalized) {
+      references.push(normalized);
+    }
+  }
+  return references;
+}
 async function convertHtmlToMarkdown(html, url) {
   const dom = new JSDOM(html, { url });
   const article = new Readability(dom.window.document).parse();
@@ -304,82 +384,268 @@ async function readManifestByHash(manifestsDir, contentHash) {
   }
   return null;
 }
-async function ingestInput(rootDir, input) {
-  const { paths } = await initWorkspace(rootDir);
-  await ensureDir(path4.join(paths.rawDir, "sources"));
+async function persistPreparedInput(rootDir, prepared, paths) {
+  await ensureDir(paths.rawSourcesDir);
+  await ensureDir(paths.rawAssetsDir);
   await ensureDir(paths.manifestsDir);
   await ensureDir(paths.extractsDir);
-  const isUrl = /^https?:\/\//i.test(input);
-  const now = (/* @__PURE__ */ new Date()).toISOString();
-  let title = path4.basename(input);
-  let mimeType = "application/octet-stream";
-  let storedExtension = ".bin";
-  let payloadBytes;
-  let extractedTextPath;
-  let sourceKind = "binary";
-  if (isUrl) {
-    const response = await fetch(input);
-    if (!response.ok) {
-      throw new Error(`Failed to fetch ${input}: ${response.status} ${response.statusText}`);
-    }
-    const arrayBuffer = await response.arrayBuffer();
-    payloadBytes = Buffer.from(arrayBuffer);
-    mimeType = response.headers.get("content-type")?.split(";")[0]?.trim() || guessMimeType(input);
-    sourceKind = inferKind(mimeType, input);
-    if (sourceKind === "html" || mimeType.startsWith("text/html")) {
-      const html = payloadBytes.toString("utf8");
-      const converted = await convertHtmlToMarkdown(html, input);
-      title = converted.title;
-      payloadBytes = Buffer.from(converted.markdown, "utf8");
-      mimeType = "text/markdown";
-      sourceKind = "markdown";
-      storedExtension = ".md";
-    } else {
-      title = new URL(input).hostname + new URL(input).pathname;
-      const extension = path4.extname(new URL(input).pathname) || (mime.extension(mimeType) ? `.${mime.extension(mimeType)}` : ".bin");
-      storedExtension = extension;
-    }
-  } else {
-    const absoluteInput = path4.resolve(rootDir, input);
-    payloadBytes = await fs3.readFile(absoluteInput);
-    mimeType = guessMimeType(absoluteInput);
-    sourceKind = inferKind(mimeType, absoluteInput);
-    storedExtension = path4.extname(absoluteInput) || `.${mime.extension(mimeType) || "bin"}`;
-    if (sourceKind === "markdown" || sourceKind === "text") {
-      title = titleFromText(path4.basename(absoluteInput, path4.extname(absoluteInput)), payloadBytes.toString("utf8"));
-    } else {
-      title = path4.basename(absoluteInput, path4.extname(absoluteInput));
-    }
-  }
-  const contentHash = sha256(payloadBytes);
+  const attachments = prepared.attachments ?? [];
+  const contentHash = prepared.contentHash ?? buildCompositeHash(prepared.payloadBytes, attachments);
   const existing = await readManifestByHash(paths.manifestsDir, contentHash);
   if (existing) {
-    return existing;
+    return { manifest: existing, isNew: false };
   }
-  const sourceId = `${slugify(title)}-${contentHash.slice(0, 8)}`;
-  const storedPath = path4.join(paths.rawDir, "sources", `${sourceId}${storedExtension}`);
-  await fs3.writeFile(storedPath, payloadBytes);
-  if (sourceKind === "markdown" || sourceKind === "text") {
+  const now = (/* @__PURE__ */ new Date()).toISOString();
+  const sourceId = `${slugify(prepared.title)}-${contentHash.slice(0, 8)}`;
+  const storedPath = path4.join(paths.rawSourcesDir, `${sourceId}${prepared.storedExtension}`);
+  await fs3.writeFile(storedPath, prepared.payloadBytes);
+  let extractedTextPath;
+  if (prepared.extractedText) {
     extractedTextPath = path4.join(paths.extractsDir, `${sourceId}.md`);
-    await fs3.writeFile(extractedTextPath, payloadBytes.toString("utf8"), "utf8");
+    await fs3.writeFile(extractedTextPath, prepared.extractedText, "utf8");
+  }
+  const manifestAttachments = [];
+  for (const attachment of attachments) {
+    const absoluteAttachmentPath = path4.join(paths.rawAssetsDir, sourceId, attachment.relativePath);
+    await ensureDir(path4.dirname(absoluteAttachmentPath));
+    await fs3.writeFile(absoluteAttachmentPath, attachment.bytes);
+    manifestAttachments.push({
+      path: toPosix(path4.relative(rootDir, absoluteAttachmentPath)),
+      mimeType: attachment.mimeType,
+      originalPath: attachment.originalPath
+    });
   }
   const manifest = {
     sourceId,
-    title,
-    originType: isUrl ? "url" : "file",
-    sourceKind,
-    originalPath: isUrl ? void 0 : toPosix(path4.resolve(rootDir, input)),
-    url: isUrl ? input : void 0,
+    title: prepared.title,
+    originType: prepared.originType,
+    sourceKind: prepared.sourceKind,
+    originalPath: prepared.originalPath,
+    url: prepared.url,
     storedPath: toPosix(path4.relative(rootDir, storedPath)),
     extractedTextPath: extractedTextPath ? toPosix(path4.relative(rootDir, extractedTextPath)) : void 0,
-    mimeType,
+    mimeType: prepared.mimeType,
     contentHash,
     createdAt: now,
-    updatedAt: now
+    updatedAt: now,
+    attachments: manifestAttachments.length ? manifestAttachments : void 0
   };
   await writeJsonFile(path4.join(paths.manifestsDir, `${sourceId}.json`), manifest);
-  await appendLogEntry(rootDir, "ingest", title, [`source_id=${sourceId}`, `kind=${sourceKind}`]);
-  return manifest;
+  await appendLogEntry(rootDir, "ingest", prepared.title, [
+    `source_id=${sourceId}`,
+    `kind=${prepared.sourceKind}`,
+    `attachments=${manifestAttachments.length}`
+  ]);
+  return { manifest, isNew: true };
+}
+async function prepareFileInput(rootDir, absoluteInput) {
+  const payloadBytes = await fs3.readFile(absoluteInput);
+  const mimeType = guessMimeType(absoluteInput);
+  const sourceKind = inferKind(mimeType, absoluteInput);
+  const storedExtension = path4.extname(absoluteInput) || `.${mime.extension(mimeType) || "bin"}`;
+  let title;
+  let extractedText;
+  if (sourceKind === "markdown" || sourceKind === "text") {
+    extractedText = payloadBytes.toString("utf8");
+    title = titleFromText(path4.basename(absoluteInput, path4.extname(absoluteInput)), extractedText);
+  } else {
+    title = path4.basename(absoluteInput, path4.extname(absoluteInput));
+  }
+  return {
+    title,
+    originType: "file",
+    sourceKind,
+    originalPath: toPosix(absoluteInput),
+    mimeType,
+    storedExtension,
+    payloadBytes,
+    extractedText
+  };
+}
+async function prepareUrlInput(input) {
+  const response = await fetch(input);
+  if (!response.ok) {
+    throw new Error(`Failed to fetch ${input}: ${response.status} ${response.statusText}`);
+  }
+  let payloadBytes = Buffer.from(await response.arrayBuffer());
+  let mimeType = response.headers.get("content-type")?.split(";")[0]?.trim() || guessMimeType(input);
+  let sourceKind = inferKind(mimeType, input);
+  let storedExtension = ".bin";
+  let title = new URL(input).hostname + new URL(input).pathname;
+  let extractedText;
+  if (sourceKind === "html" || mimeType.startsWith("text/html")) {
+    const html = payloadBytes.toString("utf8");
+    const converted = await convertHtmlToMarkdown(html, input);
+    title = converted.title;
+    extractedText = converted.markdown;
+    payloadBytes = Buffer.from(converted.markdown, "utf8");
+    mimeType = "text/markdown";
+    sourceKind = "markdown";
+    storedExtension = ".md";
+  } else {
+    const extension = path4.extname(new URL(input).pathname);
+    storedExtension = extension || `.${mime.extension(mimeType) || "bin"}`;
+    if (sourceKind === "markdown" || sourceKind === "text") {
+      extractedText = payloadBytes.toString("utf8");
+      title = titleFromText(title || new URL(input).hostname, extractedText);
+    }
+  }
+  return {
+    title,
+    originType: "url",
+    sourceKind,
+    url: input,
+    mimeType,
+    storedExtension,
+    payloadBytes,
+    extractedText
+  };
+}
+async function collectInboxAttachmentRefs(inputDir, files) {
+  const refsBySource = /* @__PURE__ */ new Map();
+  for (const absolutePath of files) {
+    const mimeType = guessMimeType(absolutePath);
+    const sourceKind = inferKind(mimeType, absolutePath);
+    if (sourceKind !== "markdown") {
+      continue;
+    }
+    const content = await fs3.readFile(absolutePath, "utf8");
+    const refs = extractMarkdownReferences(content);
+    if (!refs.length) {
+      continue;
+    }
+    const sourceRefs = [];
+    for (const ref of refs) {
+      const resolved = path4.resolve(path4.dirname(absolutePath), ref);
+      if (!resolved.startsWith(inputDir) || !await fileExists(resolved)) {
+        continue;
+      }
+      sourceRefs.push({
+        absolutePath: resolved,
+        relativeRef: ref
+      });
+    }
+    if (sourceRefs.length) {
+      refsBySource.set(
+        absolutePath,
+        sourceRefs.filter(
+          (ref, index, items) => index === items.findIndex((candidate) => candidate.absolutePath === ref.absolutePath && candidate.relativeRef === ref.relativeRef)
+        )
+      );
+    }
+  }
+  return refsBySource;
+}
+function rewriteMarkdownReferences(content, replacements) {
+  return content.replace(/(!?\[[^\]]*]\()([^)]+)(\))/g, (fullMatch, prefix, target, suffix) => {
+    const normalized = normalizeLocalReference(target);
+    if (!normalized) {
+      return fullMatch;
+    }
+    const replacement = replacements.get(normalized);
+    if (!replacement) {
+      return fullMatch;
+    }
+    return `${prefix}${replacement}${suffix}`;
+  });
+}
+async function prepareInboxMarkdownInput(absolutePath, attachmentRefs) {
+  const originalBytes = await fs3.readFile(absolutePath);
+  const originalText = originalBytes.toString("utf8");
+  const title = titleFromText(path4.basename(absolutePath, path4.extname(absolutePath)), originalText);
+  const attachments = [];
+  for (const attachmentRef of attachmentRefs) {
+    const bytes = await fs3.readFile(attachmentRef.absolutePath);
+    attachments.push({
+      relativePath: sanitizeAssetRelativePath(attachmentRef.relativeRef),
+      mimeType: guessMimeType(attachmentRef.absolutePath),
+      originalPath: toPosix(attachmentRef.absolutePath),
+      bytes
+    });
+  }
+  const contentHash = buildCompositeHash(originalBytes, attachments);
+  const sourceId = `${slugify(title)}-${contentHash.slice(0, 8)}`;
+  const replacements = new Map(
+    attachmentRefs.map((attachmentRef) => [
+      attachmentRef.relativeRef.replace(/\\/g, "/"),
+      `../assets/${sourceId}/${sanitizeAssetRelativePath(attachmentRef.relativeRef)}`
+    ])
+  );
+  const rewrittenText = rewriteMarkdownReferences(originalText, replacements);
+  return {
+    title,
+    originType: "file",
+    sourceKind: "markdown",
+    originalPath: toPosix(absolutePath),
+    mimeType: "text/markdown",
+    storedExtension: path4.extname(absolutePath) || ".md",
+    payloadBytes: Buffer.from(rewrittenText, "utf8"),
+    extractedText: rewrittenText,
+    attachments,
+    contentHash
+  };
+}
+function isSupportedInboxKind(sourceKind) {
+  return ["markdown", "text", "html", "pdf", "image"].includes(sourceKind);
+}
+async function ingestInput(rootDir, input) {
+  const { paths } = await initWorkspace(rootDir);
+  const prepared = /^https?:\/\//i.test(input) ? await prepareUrlInput(input) : await prepareFileInput(rootDir, path4.resolve(rootDir, input));
+  const result = await persistPreparedInput(rootDir, prepared, paths);
+  return result.manifest;
+}
+async function importInbox(rootDir, inputDir) {
+  const { paths } = await initWorkspace(rootDir);
+  const effectiveInputDir = path4.resolve(rootDir, inputDir ?? paths.inboxDir);
+  if (!await fileExists(effectiveInputDir)) {
+    throw new Error(`Inbox directory not found: ${effectiveInputDir}`);
+  }
+  const files = (await listFilesRecursive(effectiveInputDir)).sort();
+  const refsBySource = await collectInboxAttachmentRefs(effectiveInputDir, files);
+  const claimedAttachments = new Set(
+    [...refsBySource.values()].flatMap((refs) => refs.map((ref) => ref.absolutePath))
+  );
+  const imported = [];
+  const skipped = [];
+  let attachmentCount = 0;
+  for (const absolutePath of files) {
+    const basename = path4.basename(absolutePath);
+    if (basename.startsWith(".")) {
+      skipped.push({ path: toPosix(path4.relative(rootDir, absolutePath)), reason: "hidden_file" });
+      continue;
+    }
+    if (claimedAttachments.has(absolutePath)) {
+      skipped.push({ path: toPosix(path4.relative(rootDir, absolutePath)), reason: "referenced_attachment" });
+      continue;
+    }
+    const mimeType = guessMimeType(absolutePath);
+    const sourceKind = inferKind(mimeType, absolutePath);
+    if (!isSupportedInboxKind(sourceKind)) {
+      skipped.push({ path: toPosix(path4.relative(rootDir, absolutePath)), reason: `unsupported_kind:${sourceKind}` });
+      continue;
+    }
+    const prepared = sourceKind === "markdown" && refsBySource.has(absolutePath) ? await prepareInboxMarkdownInput(absolutePath, refsBySource.get(absolutePath) ?? []) : await prepareFileInput(rootDir, absolutePath);
+    const result = await persistPreparedInput(rootDir, prepared, paths);
+    if (!result.isNew) {
+      skipped.push({ path: toPosix(path4.relative(rootDir, absolutePath)), reason: "duplicate_content" });
+      continue;
+    }
+    attachmentCount += result.manifest.attachments?.length ?? 0;
+    imported.push(result.manifest);
+  }
+  await appendLogEntry(rootDir, "inbox_import", toPosix(path4.relative(rootDir, effectiveInputDir)) || ".", [
+    `scanned=${files.length}`,
+    `imported=${imported.length}`,
+    `attachments=${attachmentCount}`,
+    `skipped=${skipped.length}`
+  ]);
+  return {
+    inputDir: effectiveInputDir,
+    scannedCount: files.length,
+    attachmentCount,
+    imported,
+    skipped
+  };
 }
 async function listManifests(rootDir) {
   const { paths } = await loadVaultConfig(rootDir);
@@ -1550,6 +1816,49 @@ ${excerpts.join("\n\n---\n\n")}`
   await appendLogEntry(rootDir, "query", question, [`citations=${citations.join(",") || "none"}`, `saved=${Boolean(savedTo)}`]);
   return { answer, savedTo, citations };
 }
+async function searchVault(rootDir, query, limit = 5) {
+  const { paths } = await loadVaultConfig(rootDir);
+  if (!await fileExists(paths.searchDbPath)) {
+    await compileVault(rootDir);
+  }
+  return searchPages(paths.searchDbPath, query, limit);
+}
+async function listPages(rootDir) {
+  const { paths } = await loadVaultConfig(rootDir);
+  const graph = await readJsonFile(paths.graphPath);
+  return graph?.pages ?? [];
+}
+async function readPage(rootDir, relativePath) {
+  const { paths } = await loadVaultConfig(rootDir);
+  const absolutePath = path9.resolve(paths.wikiDir, relativePath);
+  if (!absolutePath.startsWith(paths.wikiDir) || !await fileExists(absolutePath)) {
+    return null;
+  }
+  const raw = await fs7.readFile(absolutePath, "utf8");
+  const parsed = matter3(raw);
+  return {
+    path: relativePath,
+    title: typeof parsed.data.title === "string" ? parsed.data.title : path9.basename(relativePath, path9.extname(relativePath)),
+    frontmatter: parsed.data,
+    content: parsed.content
+  };
+}
+async function getWorkspaceInfo(rootDir) {
+  const { paths } = await loadVaultConfig(rootDir);
+  const manifests = await listManifests(rootDir);
+  const pages = await listPages(rootDir);
+  return {
+    rootDir,
+    configPath: paths.configPath,
+    rawDir: paths.rawDir,
+    wikiDir: paths.wikiDir,
+    stateDir: paths.stateDir,
+    agentDir: paths.agentDir,
+    inboxDir: paths.inboxDir,
+    sourceCount: manifests.length,
+    pageCount: pages.length
+  };
+}
 async function lintVault(rootDir) {
   const { paths } = await loadVaultConfig(rootDir);
   const manifests = await listManifests(rootDir);
@@ -1667,13 +1976,300 @@ async function startGraphServer(rootDir, port) {
     }
   };
 }
+
+// src/mcp.ts
+import fs9 from "fs/promises";
+import path11 from "path";
+import { McpServer, ResourceTemplate } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { z as z6 } from "zod";
+var SERVER_VERSION = "0.1.3";
+async function createMcpServer(rootDir) {
+  const server = new McpServer({
+    name: "swarmvault",
+    version: SERVER_VERSION,
+    websiteUrl: "https://www.swarmvault.ai"
+  });
+  server.registerTool("workspace_info", {
+    description: "Return the current SwarmVault workspace paths and high-level counts."
+  }, async () => {
+    const info = await getWorkspaceInfo(rootDir);
+    return asToolText(info);
+  });
+  server.registerTool("search_pages", {
+    description: "Search compiled wiki pages using the local full-text index.",
+    inputSchema: {
+      query: z6.string().min(1).describe("Search query"),
+      limit: z6.number().int().min(1).max(25).optional().describe("Maximum number of results")
+    }
+  }, async ({ query, limit }) => {
+    const results = await searchVault(rootDir, query, limit ?? 5);
+    return asToolText(results);
+  });
+  server.registerTool("read_page", {
+    description: "Read a generated wiki page by its path relative to wiki/.",
+    inputSchema: {
+      path: z6.string().min(1).describe("Path relative to wiki/, for example sources/example.md")
+    }
+  }, async ({ path: relativePath }) => {
+    const page = await readPage(rootDir, relativePath);
+    if (!page) {
+      return asToolError(`Page not found: ${relativePath}`);
+    }
+    return asToolText(page);
+  });
+  server.registerTool("list_sources", {
+    description: "List source manifests in the current workspace.",
+    inputSchema: {
+      limit: z6.number().int().min(1).max(100).optional().describe("Maximum number of manifests to return")
+    }
+  }, async ({ limit }) => {
+    const manifests = await listManifests(rootDir);
+    return asToolText(limit ? manifests.slice(0, limit) : manifests);
+  });
+  server.registerTool("query_vault", {
+    description: "Ask a question against the compiled vault and optionally save the answer.",
+    inputSchema: {
+      question: z6.string().min(1).describe("Question to ask the vault"),
+      save: z6.boolean().optional().describe("Persist the answer to wiki/outputs")
+    }
+  }, async ({ question, save }) => {
+    const result = await queryVault(rootDir, question, save ?? false);
+    return asToolText(result);
+  });
+  server.registerTool("ingest_input", {
+    description: "Ingest a local file path or URL into the SwarmVault workspace.",
+    inputSchema: {
+      input: z6.string().min(1).describe("Local path or URL to ingest")
+    }
+  }, async ({ input }) => {
+    const manifest = await ingestInput(rootDir, input);
+    return asToolText(manifest);
+  });
+  server.registerTool("compile_vault", {
+    description: "Compile source manifests into wiki pages, graph data, and search index."
+  }, async () => {
+    const result = await compileVault(rootDir);
+    return asToolText(result);
+  });
+  server.registerTool("lint_vault", {
+    description: "Run anti-drift and vault health checks."
+  }, async () => {
+    const findings = await lintVault(rootDir);
+    return asToolText(findings);
+  });
+  server.registerResource("swarmvault-config", "swarmvault://config", {
+    title: "SwarmVault Config",
+    description: "The resolved SwarmVault config file.",
+    mimeType: "application/json"
+  }, async () => {
+    const { config } = await loadVaultConfig(rootDir);
+    return asTextResource("swarmvault://config", JSON.stringify(config, null, 2));
+  });
+  server.registerResource("swarmvault-graph", "swarmvault://graph", {
+    title: "SwarmVault Graph",
+    description: "The compiled graph artifact for the current workspace.",
+    mimeType: "application/json"
+  }, async () => {
+    const { paths } = await loadVaultConfig(rootDir);
+    const graph = await readJsonFile(paths.graphPath);
+    return asTextResource(
+      "swarmvault://graph",
+      JSON.stringify(graph ?? { error: "Graph artifact not found. Run `swarmvault compile` first." }, null, 2)
+    );
+  });
+  server.registerResource("swarmvault-manifests", "swarmvault://manifests", {
+    title: "SwarmVault Manifests",
+    description: "All source manifests in the workspace.",
+    mimeType: "application/json"
+  }, async () => {
+    const manifests = await listManifests(rootDir);
+    return asTextResource("swarmvault://manifests", JSON.stringify(manifests, null, 2));
+  });
+  server.registerResource(
+    "swarmvault-pages",
+    new ResourceTemplate("swarmvault://pages/{path}", {
+      list: async () => {
+        const pages = await listPages(rootDir);
+        return {
+          resources: pages.map((page) => ({
+            uri: `swarmvault://pages/${encodeURIComponent(page.path)}`,
+            name: page.title,
+            title: page.title,
+            description: `Generated ${page.kind} page`,
+            mimeType: "text/markdown"
+          }))
+        };
+      }
+    }),
+    {
+      title: "SwarmVault Pages",
+      description: "Generated wiki pages exposed as MCP resources.",
+      mimeType: "text/markdown"
+    },
+    async (_uri, variables) => {
+      const encodedPath = typeof variables.path === "string" ? variables.path : "";
+      const relativePath = decodeURIComponent(encodedPath);
+      const page = await readPage(rootDir, relativePath);
+      if (!page) {
+        return asTextResource(`swarmvault://pages/${encodedPath}`, `Page not found: ${relativePath}`);
+      }
+      const { paths } = await loadVaultConfig(rootDir);
+      const absolutePath = path11.resolve(paths.wikiDir, relativePath);
+      return asTextResource(`swarmvault://pages/${encodedPath}`, await fs9.readFile(absolutePath, "utf8"));
+    }
+  );
+  return server;
+}
+async function startMcpServer(rootDir, stdin, stdout) {
+  const server = await createMcpServer(rootDir);
+  const transport = new StdioServerTransport(stdin, stdout);
+  await server.connect(transport);
+  return {
+    close: async () => {
+      await server.close();
+    }
+  };
+}
+function asToolText(value) {
+  return {
+    content: [
+      {
+        type: "text",
+        text: JSON.stringify(value, null, 2)
+      }
+    ]
+  };
+}
+function asToolError(message) {
+  return {
+    isError: true,
+    content: [
+      {
+        type: "text",
+        text: message
+      }
+    ]
+  };
+}
+function asTextResource(uri, text) {
+  return {
+    contents: [
+      {
+        uri,
+        text
+      }
+    ]
+  };
+}
+
+// src/watch.ts
+import path12 from "path";
+import chokidar from "chokidar";
+async function watchVault(rootDir, options = {}) {
+  const { paths } = await initWorkspace(rootDir);
+  const debounceMs = options.debounceMs ?? 900;
+  let timer;
+  let running = false;
+  let pending = false;
+  let closed = false;
+  const reasons = /* @__PURE__ */ new Set();
+  const watcher = chokidar.watch(paths.inboxDir, {
+    ignoreInitial: true,
+    awaitWriteFinish: {
+      stabilityThreshold: Math.max(250, Math.floor(debounceMs / 2)),
+      pollInterval: 100
+    }
+  });
+  const schedule = (reason) => {
+    if (closed) {
+      return;
+    }
+    reasons.add(reason);
+    pending = true;
+    if (timer) {
+      clearTimeout(timer);
+    }
+    timer = setTimeout(() => {
+      void runCycle();
+    }, debounceMs);
+  };
+  const runCycle = async () => {
+    if (running || closed || !pending) {
+      return;
+    }
+    pending = false;
+    running = true;
+    const startedAt = /* @__PURE__ */ new Date();
+    const runReasons = [...reasons];
+    reasons.clear();
+    let importedCount = 0;
+    let scannedCount = 0;
+    let attachmentCount = 0;
+    let changedPages = [];
+    let lintFindingCount;
+    let success = true;
+    let error;
+    try {
+      const imported = await importInbox(rootDir, paths.inboxDir);
+      importedCount = imported.imported.length;
+      scannedCount = imported.scannedCount;
+      attachmentCount = imported.attachmentCount;
+      const compile = await compileVault(rootDir);
+      changedPages = compile.changedPages;
+      if (options.lint) {
+        const findings = await lintVault(rootDir);
+        lintFindingCount = findings.length;
+      }
+    } catch (caught) {
+      success = false;
+      error = caught instanceof Error ? caught.message : String(caught);
+    } finally {
+      const finishedAt = /* @__PURE__ */ new Date();
+      await appendWatchRun(rootDir, {
+        startedAt: startedAt.toISOString(),
+        finishedAt: finishedAt.toISOString(),
+        durationMs: finishedAt.getTime() - startedAt.getTime(),
+        inputDir: paths.inboxDir,
+        reasons: runReasons,
+        importedCount,
+        scannedCount,
+        attachmentCount,
+        changedPages,
+        lintFindingCount,
+        success,
+        error
+      });
+      running = false;
+      if (pending && !closed) {
+        schedule("queued");
+      }
+    }
+  };
+  watcher.on("add", (filePath) => schedule(`add:${toWatchReason(paths.inboxDir, filePath)}`)).on("change", (filePath) => schedule(`change:${toWatchReason(paths.inboxDir, filePath)}`)).on("unlink", (filePath) => schedule(`unlink:${toWatchReason(paths.inboxDir, filePath)}`)).on("addDir", (dirPath) => schedule(`addDir:${toWatchReason(paths.inboxDir, dirPath)}`)).on("unlinkDir", (dirPath) => schedule(`unlinkDir:${toWatchReason(paths.inboxDir, dirPath)}`)).on("error", (caught) => schedule(`error:${caught instanceof Error ? caught.message : String(caught)}`));
+  return {
+    close: async () => {
+      closed = true;
+      if (timer) {
+        clearTimeout(timer);
+      }
+      await watcher.close();
+    }
+  };
+}
+function toWatchReason(baseDir, targetPath) {
+  return path12.relative(baseDir, targetPath) || ".";
+}
 export {
   assertProviderCapability,
   bootstrapDemo,
   compileVault,
+  createMcpServer,
   createProvider,
   defaultVaultConfig,
   getProviderForTask,
+  getWorkspaceInfo,
+  importInbox,
   ingestInput,
   initVault,
   initWorkspace,
@@ -1681,9 +2277,14 @@ export {
   installConfiguredAgents,
   lintVault,
   listManifests,
+  listPages,
   loadVaultConfig,
   queryVault,
   readExtractedText,
+  readPage,
   resolvePaths,
-  startGraphServer
+  searchVault,
+  startGraphServer,
+  startMcpServer,
+  watchVault
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@swarmvaultai/engine",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Core engine for SwarmVault: ingest, compile, query, lint, and provider abstractions.",
   "type": "module",
   "main": "dist/index.js",
@@ -18,7 +18,7 @@
     "type": "git",
     "url": "git+https://github.com/swarmclawai/swarmvault.git"
   },
-  "homepage": "https://swarmvault.ai",
+  "homepage": "https://www.swarmvault.ai",
   "bugs": {
     "url": "https://github.com/swarmclawai/swarmvault/issues"
   },
@@ -40,7 +40,9 @@
     "node": ">=24.0.0"
   },
   "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.29.0",
     "@mozilla/readability": "^0.6.0",
+    "chokidar": "^4.0.3",
     "gray-matter": "^4.0.3",
     "jsdom": "^27.0.0",
     "mime-types": "^3.0.1",