@swarmvaultai/cli 3.5.0 → 3.6.0

package/README.md

@@ -2,7 +2,7 @@
 
 `@swarmvaultai/cli` is the global command-line entry point for SwarmVault.
 
-It gives you the `swarmvault` command for building a local-first knowledge vault from files, audio transcripts, YouTube URLs, reStructuredText and DOCX documents, browser clips, saved query outputs, and guided exploration runs.
+It gives you the `swarmvault` command for building a local-first knowledge vault from files, audio/video transcripts, YouTube URLs, reStructuredText and DOCX documents, browser clips, saved query outputs, and guided exploration runs.
 
 ## Install
 
@@ -35,6 +35,7 @@ sed -n '1,120p' swarmvault.schema.md
 swarmvault ingest ./notes.md
 swarmvault ingest ./customer-call.mp3
 swarmvault ingest https://www.youtube.com/watch?v=dQw4w9WgXcQ
+swarmvault ingest --video https://example.com/product-demo.mp4
 swarmvault ingest ./repo
 swarmvault add https://arxiv.org/abs/2401.12345
 swarmvault compile --max-tokens 120000
@@ -52,7 +53,9 @@ swarmvault query "Turn this into slides" --format slides
 swarmvault explore "What should I research next?" --steps 3
 swarmvault lint --deep
 swarmvault graph blast ./src/index.ts
+swarmvault graph status ./src
 swarmvault graph update .
+swarmvault graph cluster
 swarmvault graph query "Which nodes bridge the biggest clusters?"
 swarmvault graph explain "concept:drift"
 swarmvault watch status
@@ -84,6 +87,8 @@ Create a workspace with:
 
 The schema file is the vault-specific instruction layer. Edit it to define naming rules, categories, grounding expectations, and exclusions before a serious compile.
 
+Set `SWARMVAULT_OUT=<dir>` when generated artifacts should be isolated from the project root. Config and schema files stay at the root; relative `raw/`, `wiki/`, `state/`, `agent/`, and `inbox/` workspace directories resolve under the output root.
+
 `--profile` accepts `default`, `personal-research`, or a comma-separated preset list such as `reader,timeline`. For fully custom vault behavior, edit the `profile` block in `swarmvault.config.json`; that deterministic profile layer works alongside the human-written `swarmvault.schema.md`. The `personal-research` preset also sets `profile.guidedIngestDefault: true` and `profile.deepLintDefault: true`, so guided ingest/source and lint flows are on by default until you override them with `--no-guide` or `--no-deep`.
 
 ### `swarmvault scan <directory> [--port <port>] [--no-serve]`
@@ -171,15 +176,16 @@ Source-scoped artifacts are intentionally split by role:
 Ingest a local file path, directory path, or URL into immutable source storage and write manifests to `state/manifests/`.
 
 - local directories recurse by default
-- directory ingest respects `.gitignore` unless you pass `--no-gitignore`
+- directory ingest respects `.gitignore` and `.swarmvaultignore` unless you pass `--no-gitignore` or `--no-swarmvaultignore`
 - repo-aware directory ingest records `repoRelativePath` and later compile writes `state/code-index.json`
 - use `source add` instead when the same local directory, public GitHub repo root, or docs hub should stay registered and reloadable
 - URL ingest still localizes remote image references by default
 - YouTube URLs short-circuit to direct transcript capture instead of generic HTML fetch
-- local file and archive ingest supports markdown, text, reStructuredText, HTML, PDF, Word, RTF, OpenDocument, EPUB, CSV/TSV, Excel, PowerPoint, Jupyter notebooks, BibTeX, Org-mode, AsciiDoc, transcripts, Slack exports, email, calendar, audio, structured config/data, developer manifests, images, and code
+- public video URLs passed with `--video` use `yt-dlp` for audio extraction before provider-backed transcription
+- local file and archive ingest supports markdown, text, reStructuredText, HTML, PDF, Word, RTF, OpenDocument, EPUB, CSV/TSV, Excel, PowerPoint, Jupyter notebooks, BibTeX, Org-mode, AsciiDoc, transcripts, Slack exports, email, calendar, audio, video, structured config/data, developer manifests, images, and code
 - add `--guide` when you want a resumable source session, source brief, source review, source guide, and approval-bundled canonical page edits when `profile.guidedSessionMode` is `canonical_review`, with `wiki/insights/` fallback for `insights_only`
 - set `profile.guidedIngestDefault: true` when guided mode should be the default for `ingest`, and use `--no-guide` to force a plain ingest for one run
-- code-aware directory ingest currently covers JavaScript, JSX, TypeScript, TSX, Bash/shell scripts, Python, Go, Rust, Java, Kotlin, Scala, Dart, Lua, Zig, C#, C, C++, PHP, Ruby, PowerShell, Elixir, OCaml, Objective-C, ReScript, Solidity, HTML, CSS, and Vue single-file components
+- code-aware directory ingest currently covers JavaScript, JSX, TypeScript, TSX, Bash/shell scripts, Python, Go, Rust, Java, Kotlin, Scala, Dart, Lua, Zig, C#, C, C++, PHP, Ruby, PowerShell, Elixir, OCaml, Objective-C, ReScript, Solidity, HTML, CSS, Vue single-file components, and SQL files with table/view graph relations
 
 Useful flags:
 
@@ -193,6 +199,8 @@ Useful flags:
 - `--include-resources`
 - `--include-generated`
 - `--no-gitignore`
+- `--no-swarmvaultignore`
+- `--video`
 - `--no-include-assets`
 - `--max-asset-size <bytes>`
 - `--commit`
@@ -201,7 +209,7 @@ Repo ingest defaults to `first_party` material. The extra `--include-*` flags op
 
 Large repo ingest now emits low-noise progress on materially large batches, and parser compatibility failures stay local to the affected source instead of aborting unrelated analysis.
 
-Audio files use `tasks.audioProvider` when you configure a provider with `audio` capability. When no such provider is configured, SwarmVault still ingests the source and records an explicit extraction warning instead of failing. YouTube transcript ingest does not require a model provider.
+Audio and video files use `tasks.audioProvider` when you configure a provider with `audio` capability. Local video extraction shells out to `ffmpeg`; public video URL extraction with `--video` shells out to `yt-dlp`. When no audio provider or extractor binary is configured, SwarmVault still ingests the source and records an explicit extraction warning instead of failing. YouTube transcript ingest does not require a model provider.
 
 When `--commit` is set, SwarmVault stages `wiki/` and `state/` changes and creates a git commit when the vault root is inside a git worktree. Outside git, it becomes a no-op instead of failing.
 
@@ -215,6 +223,7 @@ Capture supported URLs through a normalized markdown layer before ingesting them
 - X/Twitter URLs use a graceful public capture path
 - unsupported URLs fall back to generic URL ingest instead of failing
 - optional metadata: `--author <name>` and `--contributor <name>`
+- `--video` treats the URL as a public video and routes extracted audio through `tasks.audioProvider`
 - normalized captures record fields such as `source_type`, `source_url`, `canonical_url`, `title`, `authors`, `published_at`, `updated_at`, `doi`, and `tags` when available
 - use `source add` instead when the URL is a public GitHub repo root or a docs hub that should stay synced over time
 
@@ -383,6 +392,27 @@ Refresh code-derived graph artifacts from tracked repo roots or one explicit rep
 - with `path`, refreshes that repo root instead of the tracked set
 - `--json` returns the same one-shot watch result shape, including repo import/update/remove counts and pending semantic refresh entries
 
+### `swarmvault graph status [path]`
+
+Read-only graph freshness check for tracked repo roots or one explicit repo path.
+
+- reports graph and graph-report presence
+- lists tracked repo roots and changed files without writing watch status
+- separates code-only changes from semantic refresh changes
+- recommends `swarmvault graph update` for code-only graph drift
+- recommends `swarmvault compile` when graph/report artifacts are missing, non-code files changed, or a semantic refresh is pending
+- supports global `--json` for automation
+
+### `swarmvault graph cluster [--resolution <n>]`
+
+Recompute communities, node degrees, bridge scores, god-node flags, and graph report artifacts from the existing `state/graph.json` without re-ingesting or re-analyzing sources.
+
+- writes refreshed graph metrics back to `state/graph.json`
+- updates `wiki/graph/report.md`, `wiki/graph/report.json`, share artifacts, and per-community graph pages
+- uses `graph.communityResolution` by default, or `--resolution <n>` for a one-off override
+- splits oversized or low-cohesion communities after the initial Louvain pass so large-repo reports stay scannable
+- `--json` returns counts plus the graph/report paths
+
 ### `swarmvault hook install|uninstall|status`
 
 Manage SwarmVault's local git hook blocks for the nearest git repository.
@@ -428,6 +458,7 @@ Run SwarmVault as a local MCP server over stdio. This exposes the vault to compa
 - `query_graph`
 - `graph_report`
 - `graph_stats`
+- `cluster_graph`
 - `get_node`
 - `get_community`
 - `get_neighbors`
@@ -435,7 +466,7 @@ Run SwarmVault as a local MCP server over stdio. This exposes the vault to compa
 - `shortest_path`
 - `god_nodes`
 
-`compile_vault` also accepts `maxTokens` for bounded wiki output, `graph_stats` returns lightweight graph counts, `get_community` resolves community members and pages, `blast_radius` traces reverse import impact for a file or module target, `build_context_pack` creates the same bounded agent evidence bundles as `swarmvault context build`, the task tools mirror `swarmvault task`, the memory tools mirror the compatibility command group, `doctor_vault` mirrors `swarmvault doctor`, and retrieval tools inspect or repair the local index.
+`compile_vault` also accepts `maxTokens` for bounded wiki output, `graph_stats` returns lightweight graph counts, `cluster_graph` mirrors `swarmvault graph cluster`, `get_community` resolves community members and pages, `blast_radius` traces reverse import impact for a file or module target, `build_context_pack` creates the same bounded agent evidence bundles as `swarmvault context build`, the task tools mirror `swarmvault task`, the memory tools mirror the compatibility command group, `doctor_vault` mirrors `swarmvault doctor`, and retrieval tools inspect or repair the local index.
 
 The MCP surface also exposes `swarmvault://schema`, `swarmvault://sessions`, `swarmvault://sessions/{path}`, `swarmvault://context-packs`, `swarmvault://tasks`, `swarmvault://memory-tasks`, and includes `schemaPath` in `workspace_info`.
 
@@ -523,7 +554,7 @@ Defaults:
 - namespaces every remote record by `vaultId` so multiple vaults can safely share one Neo4j database
 - upserts current graph records and does not prune stale remote data yet
 
-### `swarmvault install --agent <codex|claude|cursor|goose|pi|gemini|opencode|aider|copilot|trae|claw|droid>`
+### `swarmvault install --agent <agent>`
 
 Install agent-specific rules into the current project so an agent understands the SwarmVault workspace contract and workflow.
 
@@ -548,6 +579,10 @@ Agent target mapping:
 - `trae` writes `.trae/rules/swarmvault.md`
 - `claw` writes `.claw/skills/swarmvault/SKILL.md`
 - `droid` writes `.factory/rules/swarmvault.md`
+- `kiro` writes `.kiro/skills/swarmvault/SKILL.md` and `.kiro/steering/swarmvault.md`
+- `hermes` writes `~/.hermes/skills/swarmvault/SKILL.md` plus `AGENTS.md`
+- `antigravity` writes `.agents/rules/swarmvault.md` and `.agents/workflows/swarmvault.md`, and removes older fully managed `.agent/` files during reinstall
+- `vscode` writes `.github/chatmodes/swarmvault.chatmode.md` plus `.github/copilot-instructions.md`
 
 Hook semantics:
 

package/dist/index.js

@@ -34,6 +34,7 @@ import {
   exportObsidianVault,
   finishMemoryTask,
   getGitHookStatus,
+  getGraphStatus,
   getRetrievalStatus,
   getWatchStatus,
   graphDiff,
@@ -67,6 +68,7 @@ import {
   readGraphReport,
   readMemoryTask,
   rebuildRetrievalIndex,
+  refreshGraphClusters,
   registerLocalWhisperProvider,
   rejectApproval,
   reloadManagedSources,
@@ -309,9 +311,9 @@ program.name("swarmvault").description("SwarmVault is a local-first knowledge co
 function readCliVersion() {
   try {
     const packageJson = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8"));
-    return typeof packageJson.version === "string" && packageJson.version.trim() ? packageJson.version : "3.5.0";
+    return typeof packageJson.version === "string" && packageJson.version.trim() ? packageJson.version : "3.6.0";
   } catch {
-    return "3.5.0";
+    return "3.6.0";
   }
 }
 function parsePositiveInt(value, fallback) {
@@ -319,6 +321,11 @@ function parsePositiveInt(value, fallback) {
   const parsed = Number.parseInt(value, 10);
   return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
 }
+function parsePositiveNumber(value) {
+  if (value === void 0) return void 0;
+  const parsed = Number.parseFloat(value);
+  return Number.isFinite(parsed) && parsed > 0 ? parsed : void 0;
+}
 function collectRepeated(value, previous) {
   return [...previous, value];
 }
@@ -506,7 +513,7 @@ program.command("init").description("Initialize a SwarmVault workspace in the cu
     log(options.lite ? "Initialized SwarmVault lite workspace." : "Initialized SwarmVault workspace.");
   }
 });
-program.command("ingest").description("Ingest a local file path, directory path, or URL into the raw SwarmVault workspace.").argument("<input>", "Local file path, directory path, or URL").option("--review", "Stage a source review artifact after ingest and compile", false).option("--guide", "Stage a guided source integration bundle after ingest and compile (default: from config)").option("--no-guide", "Skip guided mode even if enabled in config").option("--answers-file <path>", "JSON file with guided-session answers keyed by question id or listed in prompt order").option("--include-assets", "Download remote image assets when ingesting URLs", true).option("--no-include-assets", "Skip downloading remote image assets when ingesting URLs").option("--max-asset-size <bytes>", "Maximum number of bytes to fetch for a single remote image asset").option("--repo-root <path>", "Override the detected repo root when ingesting a directory").option("--include <glob...>", "Only ingest files matching one or more glob patterns").option("--exclude <glob...>", "Skip files matching one or more glob patterns").option("--max-files <n>", "Maximum number of files to ingest from a directory").option("--include-third-party", "Also ingest repo files classified as third-party", false).option("--include-resources", "Also ingest repo files classified as resources", false).option("--include-generated", "Also ingest repo files classified as generated output", false).option("--no-gitignore", "Ignore .gitignore rules when ingesting a directory").option("--resume <run-id>", "Re-run only the failed files from a prior ingest run id").option("--commit", "Auto-commit wiki and state changes after ingest").option("--no-redact", "Skip PII/secret redaction for this run (overrides config)").action(
+program.command("ingest").description("Ingest a local file path, directory path, or URL into the raw SwarmVault workspace.").argument("<input>", "Local file path, directory path, or URL").option("--review", "Stage a source review artifact after ingest and compile", false).option("--guide", "Stage a guided source integration bundle after ingest and compile (default: from config)").option("--no-guide", "Skip guided mode even if enabled in config").option("--answers-file <path>", "JSON file with guided-session answers keyed by question id or listed in prompt order").option("--include-assets", "Download remote image assets when ingesting URLs", true).option("--no-include-assets", "Skip downloading remote image assets when ingesting URLs").option("--max-asset-size <bytes>", "Maximum number of bytes to fetch for a single remote image asset").option("--repo-root <path>", "Override the detected repo root when ingesting a directory").option("--include <glob...>", "Only ingest files matching one or more glob patterns").option("--exclude <glob...>", "Skip files matching one or more glob patterns").option("--max-files <n>", "Maximum number of files to ingest from a directory").option("--include-third-party", "Also ingest repo files classified as third-party", false).option("--include-resources", "Also ingest repo files classified as resources", false).option("--include-generated", "Also ingest repo files classified as generated output", false).option("--no-gitignore", "Ignore .gitignore rules when ingesting a directory").option("--no-swarmvaultignore", "Ignore .swarmvaultignore rules when ingesting a directory").option("--video", "Treat a URL input as a public video and transcribe extracted audio", false).option("--resume <run-id>", "Re-run only the failed files from a prior ingest run id").option("--commit", "Auto-commit wiki and state changes after ingest").option("--no-redact", "Skip PII/secret redaction for this run (overrides config)").action(
   async (input, options) => {
     const guideAnswers = readGuideAnswersFile(options.answersFile);
     const vaultConfig = await loadVaultConfig(process2.cwd()).catch(() => null);
@@ -527,6 +534,8 @@ program.command("ingest").description("Ingest a local file path, directory path,
       exclude: options.exclude,
       maxFiles,
       gitignore: options.gitignore,
+      swarmvaultignore: options.swarmvaultignore,
+      video: options.video,
       extractClasses,
       resume: options.resume,
       redact: options.redact
@@ -643,10 +652,11 @@ program.command("ingest").description("Ingest a local file path, directory path,
     }
   }
 );
-program.command("add").description("Capture supported URLs into normalized markdown before ingesting them.").argument("<input>", "Supported URL or bare arXiv id").option("--author <name>", "Human author or curator for this capture").option("--contributor <name>", "Additional contributor metadata for this capture").option("--no-redact", "Skip PII/secret redaction for this capture (overrides config)").action(async (input, options) => {
+program.command("add").description("Capture supported URLs into normalized markdown before ingesting them.").argument("<input>", "Supported URL or bare arXiv id").option("--author <name>", "Human author or curator for this capture").option("--contributor <name>", "Additional contributor metadata for this capture").option("--video", "Treat the URL as a public video and transcribe extracted audio", false).option("--no-redact", "Skip PII/secret redaction for this capture (overrides config)").action(async (input, options) => {
   const result = await addInput(process2.cwd(), input, {
     author: options.author,
     contributor: options.contributor,
+    video: options.video,
     redact: options.redact
   });
   if (isJson()) {
@@ -1150,6 +1160,46 @@ graph.command("update").alias("refresh").description("Refresh code-derived graph
     `Updated graph from ${result.watchedRepoRoots.length} repo root${result.watchedRepoRoots.length === 1 ? "" : "s"}. Imported ${result.repoImportedCount}, updated ${result.repoUpdatedCount}, removed ${result.repoRemovedCount}, pending semantic refresh ${result.pendingSemanticRefreshCount}.`
   );
 });
+graph.command("status").description("Read-only check for graph/report presence and tracked repo changes.").argument("[path]", "Optional repo root to check instead of configured/tracked roots").action(async (targetPath) => {
+  const overrideRoots = targetPath ? [path2.resolve(process2.cwd(), targetPath)] : void 0;
+  const status = await getGraphStatus(process2.cwd(), { repoRoots: overrideRoots });
+  if (isJson()) {
+    emitJson(status);
+    return;
+  }
+  log(`Graph: ${status.graphExists ? status.graphPath : `missing (${status.graphPath})`}`);
+  log(`Report: ${status.reportExists ? status.reportPath : `missing (${status.reportPath})`}`);
+  log(`Tracked repo roots: ${status.trackedRepoRoots.length ? status.trackedRepoRoots.join(", ") : "none"}`);
+  log(`Code changes: ${status.codeChangeCount}`);
+  log(`Semantic changes: ${status.semanticChangeCount}`);
+  log(`Pending semantic refresh: ${status.pendingSemanticRefresh.length}`);
+  if (status.changes.length) {
+    for (const change of status.changes.slice(0, 20)) {
+      log(`- ${change.refreshType} ${change.changeType} ${change.path}`);
+    }
+    if (status.changes.length > 20) {
+      log(`... and ${status.changes.length - 20} more`);
+    }
+  }
+  log(`State: ${status.stale ? "stale" : "fresh"}`);
+  if (status.recommendedCommand) {
+    log(`Recommended: ${status.recommendedCommand}`);
+  }
+});
+graph.command("cluster").alias("clusters").description("Recompute graph communities, degrees, god-node flags, and graph report artifacts without re-ingesting sources.").option("--resolution <number>", "Override the Louvain community resolution for this run").action(async (options) => {
+  const resolution = parsePositiveNumber(options.resolution);
+  if (options.resolution && resolution === void 0) {
+    throw new Error("--resolution must be a positive number.");
+  }
+  const result = await refreshGraphClusters(process2.cwd(), { resolution });
+  if (isJson()) {
+    emitJson(result);
+    return;
+  }
+  log(
+    `Refreshed ${result.communityCount} communities across ${result.nodeCount} nodes and ${result.edgeCount} edges. Report: ${result.reportPath}`
+  );
+});
 graphPush.command("neo4j").description("Push the compiled graph directly into Neo4j over Bolt/Aura.").option("--uri <bolt-uri>", "Neo4j Bolt or Aura URI").option("--username <user>", "Neo4j username").option("--password-env <env-var>", "Environment variable containing the Neo4j password").option("--database <name>", "Neo4j database name").option("--vault-id <id>", "Stable vault identifier used for shared-database namespacing").option("--batch-size <n>", "Maximum rows to write per Neo4j transaction batch").option("--include-third-party", "Also push third-party repo material", false).option("--include-resources", "Also push resource-like content", false).option("--include-generated", "Also push generated output", false).option("--dry-run", "Show what would be pushed without writing to Neo4j", false).action(
   async (options) => {
     const batchSize = typeof options.batchSize === "string" && options.batchSize.trim() ? parsePositiveInt(options.batchSize, 0) || void 0 : void 0;
@@ -2114,9 +2164,10 @@ program.command("scan").description("Quick-start: initialize, ingest, compile, a
     log(`Ingested ${result.imported.length} file(s).`);
   }
   const compiled = await compileVault(rootDir, {});
-  const shareCardPath = path2.join(rootDir, "wiki", "graph", "share-card.md");
-  const shareCardSvgPath = path2.join(rootDir, "wiki", "graph", "share-card.svg");
-  const shareKitPath = path2.join(rootDir, "wiki", "graph", "share-kit");
+  const { paths } = await loadVaultConfig(rootDir);
+  const shareCardPath = path2.join(paths.wikiDir, "graph", "share-card.md");
+  const shareCardSvgPath = path2.join(paths.wikiDir, "graph", "share-card.svg");
+  const shareKitPath = path2.join(paths.wikiDir, "graph", "share-kit");
   if (!isJson()) {
     log(`Compiled ${compiled.sourceCount} source(s), ${compiled.pageCount} page(s).`);
     log(`Share card: ${shareCardPath}`);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@swarmvaultai/cli",
-  "version": "3.5.0",
+  "version": "3.6.0",
   "description": "Global CLI for SwarmVault.",
   "type": "module",
   "main": "dist/index.js",
@@ -44,7 +44,7 @@
     "prepublishOnly": "node ../../scripts/check-release-sync.mjs && node ../../scripts/check-published-manifests.mjs"
   },
   "dependencies": {
-    "@swarmvaultai/engine": "3.5.0",
+    "@swarmvaultai/engine": "3.6.0",
     "commander": "^14.0.1"
   },
   "devDependencies": {