@swarmvaultai/engine 0.1.2 → 0.1.4

package/README.md

@@ -2,7 +2,7 @@
 
 `@swarmvaultai/engine` is the runtime library behind SwarmVault.
 
-It provides the ingest, compile, query, lint, provider, graph, search, and viewer-serving primitives used by the CLI.
+It exposes the primitives for initializing a workspace, ingesting sources, importing an inbox, compiling a wiki, querying the vault, running lint, serving the graph viewer, watching the inbox, and exposing the vault over MCP.
 
 ## Who This Is For
 
@@ -10,7 +10,8 @@ Use this package if you want to:
 
 - build your own interface on top of the SwarmVault runtime
 - integrate vault operations into another Node application
-- customize provider loading or workspace orchestration without shelling out to the CLI
+- embed watch or MCP behavior without shelling out to the CLI
+- customize provider loading or orchestration in code
 
 If you only want to use SwarmVault as a tool, install `@swarmvaultai/cli` instead.
 
@@ -19,48 +20,70 @@ If you only want to use SwarmVault as a tool, install `@swarmvaultai/cli` instea
 ```ts
 import {
   compileVault,
-  createProvider,
+  createMcpServer,
   defaultVaultConfig,
+  defaultVaultSchema,
+  importInbox,
   ingestInput,
   initVault,
   installAgent,
   lintVault,
   loadVaultConfig,
+  loadVaultSchema,
   queryVault,
-  startGraphServer
+  searchVault,
+  startGraphServer,
+  startMcpServer,
+  watchVault,
 } from "@swarmvaultai/engine";
 ```
 
-The engine also exports the main runtime types for providers, graph artifacts, pages, manifests, and lint findings.
+The engine also exports the main runtime types for providers, graph artifacts, pages, manifests, query results, lint findings, and watch records.
 
 ## Example
 
 ```ts
-import { compileVault, ingestInput, initVault, queryVault } from "@swarmvaultai/engine";
+import { compileVault, importInbox, initVault, loadVaultSchema, queryVault, watchVault } from "@swarmvaultai/engine";
 
 const rootDir = process.cwd();
 
 await initVault(rootDir);
-await ingestInput(rootDir, "./research.md");
+const schema = await loadVaultSchema(rootDir);
+console.log(schema.path);
+await importInbox(rootDir);
 await compileVault(rootDir);
 
 const result = await queryVault(rootDir, "What changed most recently?", true);
 console.log(result.answer);
+
+const watcher = await watchVault(rootDir, { lint: true });
 ```
 
+## Schema Layer
+
+Each workspace carries a root markdown file named `swarmvault.schema.md`.
+
+The engine treats that file as vault-specific operating guidance for compile and query work. In `v0.1.4`:
+
+- `initVault()` creates the default schema file
+- `loadVaultSchema()` resolves the canonical file and legacy `schema.md` fallback
+- compile and query prompts include the schema content
+- generated pages store `schema_hash`
+- `lintVault()` marks generated pages stale when the schema changes
+
 ## Provider Model
 
 The engine supports:
 
+- `heuristic`
 - `openai`
-- `ollama`
 - `anthropic`
 - `gemini`
+- `ollama`
 - `openai-compatible`
 - `custom`
-- `heuristic`
 
-Providers are validated through capabilities such as:
+Providers are capability-driven. Each provider declares support for features such as:
 
 - `chat`
 - `structured`
@@ -70,29 +93,57 @@ Providers are validated through capabilities such as:
 - `streaming`
 - `local`
 
-This matters because many "OpenAI-compatible" providers implement only part of the OpenAI surface. The engine is designed to route by capability, not by brand name.
+This matters because many "OpenAI-compatible" backends only implement part of the OpenAI surface.
 
-## Artifacts
+## Main Engine Surfaces
 
-Running the engine produces a local workspace with four main areas:
+### Ingest
 
-- `raw/`: immutable source inputs and captured assets
-- `wiki/`: generated markdown pages and saved outputs
-- `state/`: manifests, extracts, graph state, compile state, and search index
-- `agent/`: generated agent rules
+- `ingestInput(rootDir, input)` ingests a local path or URL
+- `importInbox(rootDir, inputDir?)` recursively imports supported inbox files and browser-clipper style bundles
+
+### Compile + Query
 
-Important artifacts include:
+- `compileVault(rootDir)` writes wiki pages, graph data, and search state using the vault schema as guidance
+- `queryVault(rootDir, question, save)` answers against the compiled vault using the same schema layer
+- `searchVault(rootDir, query, limit)` searches compiled pages directly
 
-- `state/graph.json`
-- `state/search.sqlite`
-- `wiki/index.md`
-- `wiki/sources/*.md`
-- `wiki/concepts/*.md`
-- `wiki/entities/*.md`
-- `wiki/outputs/*.md`
+### Automation
+
+- `watchVault(rootDir, options)` watches the inbox and appends run records to `state/jobs.ndjson`
+- `lintVault(rootDir)` runs health and anti-drift checks
+
+### MCP
+
+- `createMcpServer(rootDir)` creates an MCP server instance
+- `startMcpServer(rootDir)` runs the MCP server over stdio
+
+The MCP surface includes tools for workspace info, page search, page reads, source listing, querying, ingestion, compile, and lint, along with resources for config, graph, manifests, schema, and page content.
+
+## Artifacts
+
+Running the engine produces a local workspace with these main areas:
+
+- `swarmvault.schema.md`: vault-specific compile and query instructions
+- `inbox/`: capture staging area for markdown bundles and imported files
+- `raw/sources/`: immutable source copies
+- `raw/assets/`: copied attachments referenced by ingested markdown bundles
+- `wiki/`: generated markdown pages and saved outputs
+- `state/manifests/`: source manifests
+- `state/extracts/`: extracted text
+- `state/analyses/`: model analysis output
+- `state/graph.json`: compiled graph
+- `state/search.sqlite`: full-text index
+- `state/jobs.ndjson`: watch-mode automation logs
 
 ## Notes
 
 - The engine expects Node `>=24`
 - The local search layer currently uses the built-in `node:sqlite` module, which may emit an experimental warning in Node 24
 - The viewer source lives in the companion `@swarmvaultai/viewer` package, and the built assets are bundled into the engine package for CLI installs
+
+## Links
+
+- Website: https://www.swarmvault.ai
+- Docs: https://www.swarmvault.ai/docs
+- GitHub: https://github.com/swarmclawai/swarmvault

package/dist/index.d.ts

@@ -1,4 +1,6 @@
 import { z } from 'zod';
+import { Readable, Writable } from 'node:stream';
+import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
 
 declare const providerCapabilitySchema: z.ZodEnum<{
     responses: "responses";
@@ -66,6 +68,7 @@ interface VaultConfig {
         wikiDir: string;
         stateDir: string;
         agentDir: string;
+        inboxDir: string;
     };
     providers: Record<string, ProviderConfig>;
     tasks: {
@@ -81,10 +84,14 @@ interface VaultConfig {
 }
 interface ResolvedPaths {
     rootDir: string;
+    schemaPath: string;
     rawDir: string;
+    rawSourcesDir: string;
+    rawAssetsDir: string;
     wikiDir: string;
     stateDir: string;
     agentDir: string;
+    inboxDir: string;
     manifestsDir: string;
     extractsDir: string;
     analysesDir: string;
@@ -92,8 +99,14 @@ interface ResolvedPaths {
     graphPath: string;
     searchDbPath: string;
     compileStatePath: string;
+    jobsLogPath: string;
     configPath: string;
 }
+interface SourceAttachment {
+    path: string;
+    mimeType: string;
+    originalPath?: string;
+}
 interface SourceManifest {
     sourceId: string;
     title: string;
@@ -107,6 +120,7 @@ interface SourceManifest {
     contentHash: string;
     createdAt: string;
     updatedAt: string;
+    attachments?: SourceAttachment[];
 }
 interface AnalyzedTerm {
     id: string;
@@ -124,6 +138,7 @@ interface SourceClaim {
 interface SourceAnalysis {
     sourceId: string;
     sourceHash: string;
+    schemaHash: string;
     title: string;
     summary: string;
     concepts: AnalyzedTerm[];
@@ -160,6 +175,7 @@ interface GraphPage {
     freshness: Freshness;
     confidence: number;
     backlinks: string[];
+    schemaHash: string;
     sourceHashes: Record<string, string>;
 }
 interface GraphArtifact {
@@ -193,9 +209,42 @@ interface LintFinding {
     message: string;
     pagePath?: string;
 }
+interface InboxImportSkip {
+    path: string;
+    reason: string;
+}
+interface InboxImportResult {
+    inputDir: string;
+    scannedCount: number;
+    attachmentCount: number;
+    imported: SourceManifest[];
+    skipped: InboxImportSkip[];
+}
+interface WatchOptions {
+    lint?: boolean;
+    debounceMs?: number;
+}
+interface WatchRunRecord {
+    startedAt: string;
+    finishedAt: string;
+    durationMs: number;
+    inputDir: string;
+    reasons: string[];
+    importedCount: number;
+    scannedCount: number;
+    attachmentCount: number;
+    changedPages: string[];
+    lintFindingCount?: number;
+    success: boolean;
+    error?: string;
+}
+interface WatchController {
+    close(): Promise<void>;
+}
 
 declare function defaultVaultConfig(): VaultConfig;
-declare function resolvePaths(rootDir: string, config?: VaultConfig, configPath?: string): ResolvedPaths;
+declare function defaultVaultSchema(): string;
+declare function resolvePaths(rootDir: string, config?: VaultConfig, configPath?: string, schemaPath?: string): ResolvedPaths;
 declare function loadVaultConfig(rootDir: string): Promise<{
     config: VaultConfig;
     paths: ResolvedPaths;
@@ -206,12 +255,33 @@ declare function initWorkspace(rootDir: string): Promise<{
 }>;
 
 declare function ingestInput(rootDir: string, input: string): Promise<SourceManifest>;
+declare function importInbox(rootDir: string, inputDir?: string): Promise<InboxImportResult>;
 declare function listManifests(rootDir: string): Promise<SourceManifest[]>;
 declare function readExtractedText(rootDir: string, manifest: SourceManifest): Promise<string | undefined>;
 
 declare function initVault(rootDir: string): Promise<void>;
 declare function compileVault(rootDir: string): Promise<CompileResult>;
 declare function queryVault(rootDir: string, question: string, save?: boolean): Promise<QueryResult>;
+declare function searchVault(rootDir: string, query: string, limit?: number): Promise<SearchResult[]>;
+declare function listPages(rootDir: string): Promise<GraphPage[]>;
+declare function readPage(rootDir: string, relativePath: string): Promise<{
+    path: string;
+    title: string;
+    frontmatter: Record<string, unknown>;
+    content: string;
+} | null>;
+declare function getWorkspaceInfo(rootDir: string): Promise<{
+    rootDir: string;
+    configPath: string;
+    schemaPath: string;
+    rawDir: string;
+    wikiDir: string;
+    stateDir: string;
+    agentDir: string;
+    inboxDir: string;
+    sourceCount: number;
+    pageCount: number;
+}>;
 declare function lintVault(rootDir: string): Promise<LintFinding[]>;
 declare function bootstrapDemo(rootDir: string, input?: string): Promise<{
     manifestId?: string;
@@ -221,13 +291,28 @@ declare function bootstrapDemo(rootDir: string, input?: string): Promise<{
 declare function installAgent(rootDir: string, agent: "codex" | "claude" | "cursor"): Promise<string>;
 declare function installConfiguredAgents(rootDir: string): Promise<string[]>;
 
+interface VaultSchema {
+    path: string;
+    content: string;
+    hash: string;
+    isLegacyPath: boolean;
+}
+declare function loadVaultSchema(rootDir: string): Promise<VaultSchema>;
+
 declare function startGraphServer(rootDir: string, port?: number): Promise<{
     port: number;
     close: () => Promise<void>;
 }>;
 
+declare function createMcpServer(rootDir: string): Promise<McpServer>;
+declare function startMcpServer(rootDir: string, stdin?: Readable, stdout?: Writable): Promise<{
+    close: () => Promise<void>;
+}>;
+
+declare function watchVault(rootDir: string, options?: WatchOptions): Promise<WatchController>;
+
 declare function createProvider(id: string, config: ProviderConfig, rootDir: string): Promise<ProviderAdapter>;
 declare function getProviderForTask(rootDir: string, task: keyof Awaited<ReturnType<typeof loadVaultConfig>>["config"]["tasks"]): Promise<ProviderAdapter>;
 declare function assertProviderCapability(provider: ProviderAdapter, capability: ProviderCapability): void;
 
-export { type AnalyzedTerm, type ClaimStatus, type CompileResult, type Freshness, type GenerationAttachment, type GenerationRequest, type GenerationResponse, type GraphArtifact, type GraphEdge, type GraphNode, type GraphPage, type LintFinding, type PageKind, type Polarity, type ProviderAdapter, type ProviderCapability, type ProviderConfig, type ProviderType, type QueryResult, type ResolvedPaths, type SearchResult, type SourceAnalysis, type SourceClaim, type SourceManifest, type VaultConfig, assertProviderCapability, bootstrapDemo, compileVault, createProvider, defaultVaultConfig, getProviderForTask, ingestInput, initVault, initWorkspace, installAgent, installConfiguredAgents, lintVault, listManifests, loadVaultConfig, providerCapabilitySchema, providerTypeSchema, queryVault, readExtractedText, resolvePaths, startGraphServer };
+export { type AnalyzedTerm, type ClaimStatus, type CompileResult, type Freshness, type GenerationAttachment, type GenerationRequest, type GenerationResponse, type GraphArtifact, type GraphEdge, type GraphNode, type GraphPage, type InboxImportResult, type InboxImportSkip, type LintFinding, type PageKind, type Polarity, type ProviderAdapter, type ProviderCapability, type ProviderConfig, type ProviderType, type QueryResult, type ResolvedPaths, type SearchResult, type SourceAnalysis, type SourceAttachment, type SourceClaim, type SourceManifest, type VaultConfig, type WatchController, type WatchOptions, type WatchRunRecord, assertProviderCapability, bootstrapDemo, compileVault, createMcpServer, createProvider, defaultVaultConfig, defaultVaultSchema, getProviderForTask, getWorkspaceInfo, importInbox, ingestInput, initVault, initWorkspace, installAgent, installConfiguredAgents, lintVault, listManifests, listPages, loadVaultConfig, loadVaultSchema, providerCapabilitySchema, providerTypeSchema, queryVault, readExtractedText, readPage, resolvePaths, searchVault, startGraphServer, startMcpServer, watchVault };