voyageai-cli 1.30.9 → 1.33.0

package/README.md

@@ -6,7 +6,7 @@
 
 [![CI](https://github.com/mrlynn/voyageai-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/mrlynn/voyageai-cli/actions/workflows/ci.yml) [![npm version](https://img.shields.io/npm/v/voyageai-cli.svg)](https://www.npmjs.com/package/voyageai-cli) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT) [![Node.js](https://img.shields.io/node/v/voyageai-cli.svg)](https://nodejs.org) [![GitHub release](https://img.shields.io/github/v/release/mrlynn/voyageai-cli?label=Desktop%20App)](https://github.com/mrlynn/voyageai-cli/releases)
 
-The fastest path from documents to semantic search. Chunk files, generate [Voyage AI](https://www.mongodb.com/docs/voyageai/) embeddings, store in [MongoDB Atlas](https://www.mongodb.com/docs/atlas/atlas-vector-search/), and query with two-stage retrieval — from the terminal, your browser, or a desktop app.
+The fastest path from documents to semantic search. Chunk files, generate [Voyage AI](https://docs.vaicli.com/) embeddings, store in [MongoDB Atlas](https://www.mongodb.com/docs/atlas/atlas-vector-search/), and query with two-stage retrieval — from the terminal, your browser, or a desktop app.
 
 > **⚠️ Disclaimer:** This is an independent, community-built tool — **not** an official product of MongoDB, Inc. or Voyage AI. See [Disclaimer](#disclaimer) for details.
 
@@ -83,6 +83,7 @@ RAG workflows from npm<br/><br/>
   - [Core Workflow](#core-workflow)
   - [Individual Commands](#individual-commands)
   - [Models & Benchmarks](#models--benchmarks)
+  - [Local Inference](#local-inference)
   - [Benchmarking Your Data](#benchmarking-your-data)
   - [Learn](#learn)
   - [Environment & Auth](#environment--auth)
@@ -428,6 +429,59 @@ vai models --benchmarks
 
 Also available: `voyage-code-3` (code), `voyage-finance-2` (finance), `voyage-law-2` (legal), `rerank-2.5` / `rerank-2.5-lite`.
 
+### Local Inference
+
+Run embeddings locally with `voyage-4-nano` -- no API key, no network, no cost.
+Nano shares the same embedding space as the Voyage 4 API models, so you can
+prototype locally and upgrade to the API when ready.
+
+**Prerequisites:** Python 3.10+ and ~700MB disk space for the model.
+
+#### Setup (one-time)
+
+```bash
+vai nano setup      # Creates venv, installs deps, downloads model
+vai nano status     # Verify everything is ready
+```
+
+#### Usage
+
+```bash
+# Embed text locally
+vai embed "What is MongoDB?" --local
+
+# Run the full pipeline locally
+vai pipeline ./docs/ --local --db myapp --collection knowledge
+
+# Bulk ingest with local embeddings
+vai ingest --file corpus.jsonl --local --db myapp --collection docs
+```
+
+#### Interactive Demo
+
+```bash
+vai demo nano       # Zero-dependency guided walkthrough
+```
+
+Covers similarity matrices, MRL dimension comparison, and interactive REPL --
+all without an API key or MongoDB connection.
+
+#### Nano Commands
+
+| Command | Description |
+|---------|-------------|
+| `vai nano setup` | Set up Python venv, install deps, download model |
+| `vai nano status` | Check local inference readiness |
+| `vai nano test` | Smoke-test local inference |
+| `vai nano info` | Show model details and cache location |
+| `vai nano clear-cache` | Remove cached model files |
+
+#### Upgrade Path
+
+Since nano shares the Voyage 4 embedding space, your local embeddings are
+compatible with `voyage-4`, `voyage-4-lite`, and `voyage-4-large`. No
+re-vectorization needed when you add an API key.
+
 ### Benchmarking Your Data
 
 Published benchmarks measure average quality across standardized datasets. `vai benchmark` measures what matters for **your** use case:
@@ -501,7 +555,7 @@ vai explain models            # How to choose the right model
 
 | Variable | Required For | Description |
 |----------|-------------|-------------|
-| `VOYAGE_API_KEY` | All embedding/reranking | [Model API key](https://www.mongodb.com/docs/voyageai/management/api-keys/) from MongoDB Atlas |
+| `VOYAGE_API_KEY` | All embedding/reranking | [Model API key](https://docs.vaicli.com/management/api-keys/) from MongoDB Atlas |
 | `MONGODB_URI` | store, search, query, pipeline, index | MongoDB Atlas connection string |
 
 Credentials resolve in order: environment variables → `.env` file → `~/.vai/config.json`.
@@ -532,6 +586,35 @@ vai config set mongodb-uri "mongodb+srv://..."
 
 Config is stored in `~/.vai/config.json`. Use `vai config get` to see all values (secrets are masked) or `vai config get <key>` for a specific value. The desktop app's Settings → Database page also reads and writes this file.
 
+#### Telemetry
+
+vai collects anonymous usage telemetry for the CLI and desktop app. On first launch, vai shows a one-time notice before any telemetry is sent. The CLI and desktop app share the same telemetry preference and notice state via `~/.vai/config.json`.
+
+Use the built-in telemetry controls:
+
+```bash
+vai telemetry
+vai telemetry off
+vai telemetry on
+vai telemetry status
+vai telemetry reset
+```
+
+You can also disable telemetry with environment variables:
+
+```bash
+export VAI_TELEMETRY=0
+export DO_NOT_TRACK=1
+```
+
+For local auditing, set:
+
+```bash
+export VAI_TELEMETRY_DEBUG=1
+```
+
+This prints telemetry payloads to `stderr` instead of sending them.
+
 ### Shell Completions
 
 ```bash
@@ -724,11 +807,49 @@ vai mcp --transport http --port 3100       # HTTP (remote, multi-client)
 
 ---
 
+## Project Structure
+
+This is a **monorepo-lite** that separates the CLI from the desktop app:
+
+```
+voyageai-cli/
+├── src/            ← Core library + CLI + web playground (npm package)
+├── electron/       ← Desktop app (distributed via GitHub Releases)
+├── docs/           ← Shared documentation
+├── test/           ← Test suites
+└── .github/
+    └── workflows/
+        ├── ci.yml           ← Tests + npm publish for CLI
+        └── release-app.yml  ← Electron builds + GitHub Releases
+```
+
+### Distribution Channels
+
+| Product | Channel | What users get |
+|---------|---------|----------------|
+| **CLI** (`vai`) | `npm install -g voyageai-cli` | Terminal tool, 22 commands, RAG pipeline |
+| **Web Playground** | `vai playground` | Runs locally, no install beyond the CLI |
+| **Desktop App** | [GitHub Releases](https://github.com/mrlynn/voyageai-cli/releases) | Standalone app, no Node required |
+
+### Development Scripts
+
+```bash
+# CLI development
+npm test                    # Run test suite
+npm run test:watch         # Watch mode
+
+# Electron app development  
+npm run app:install        # Install electron dependencies
+npm run app:start          # Launch electron app
+npm run app:dev            # Launch with DevTools
+npm run app:build          # Build for all platforms
+```
+
 ## Requirements
 
 - Node.js 18+
 - [MongoDB Atlas](https://www.mongodb.com/atlas) account (free tier works)
-- [Voyage AI model API key](https://www.mongodb.com/docs/voyageai/management/api-keys/) (created in Atlas)
+- [Voyage AI model API key](https://docs.vaicli.com/management/api-keys/) (created in Atlas)
 
 ## Author
 
@@ -740,7 +861,7 @@ This is a community tool and is not affiliated with, endorsed by, or supported b
 
 For official documentation and support:
 - **MongoDB:** [mongodb.com](https://www.mongodb.com) | [Atlas](https://www.mongodb.com/atlas) | [Support](https://support.mongodb.com)
-- **Voyage AI:** [MongoDB Voyage AI Docs](https://www.mongodb.com/docs/voyageai/)
+- **Voyage AI:** [MongoDB Voyage AI Docs](https://docs.vaicli.com/)
 
 ## License
 

package/package.json

@@ -1,7 +1,8 @@
 {
   "name": "voyageai-cli",
-  "version": "1.30.9",
+  "version": "1.33.0",
   "description": "CLI for Voyage AI embeddings, reranking, and MongoDB Atlas Vector Search",
+  "_comment": "This package contains the CLI + web playground. The electron/ directory is excluded via .npmignore and distributed via GitHub Releases.",
   "bin": {
     "vai": "./src/cli.js"
   },
@@ -35,7 +36,14 @@
   },
   "scripts": {
     "test": "node --test test/**/*.test.js",
-    "release": "./scripts/release.sh"
+    "reset": "node scripts/reset.js",
+    "version": "node scripts/sync-nano-version.js && git add src/nano/nano-bridge.py",
+    "release": "./scripts/release.sh",
+    "app:install": "cd electron && npm install",
+    "app:start": "cd electron && npm start",
+    "app:dev": "cd electron && npm run dev",
+    "app:build": "cd electron && npm run build",
+    "app:package": "cd electron && npm run package"
   },
   "engines": {
     "node": ">=20.0.0"
@@ -48,6 +56,7 @@
     "dotenv": "^17.2.3",
     "mongodb": "^6.0.0",
     "ora": "^9.1.0",
+    "pdf-parse": "^2.4.5",
     "picocolors": "^1.1.1",
     "update-notifier": "^7.3.1",
     "zod": "^4.3.6"

package/src/brand/ROBOT_EXAMPLES.md

@@ -0,0 +1,142 @@
+/**
+ * @file ROBOT_INTEGRATION_EXAMPLES.md
+ *
+ * How to wire robot moments into existing vai commands.
+ * Copy the relevant snippets into each command file.
+ */
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 1. vai --help  (src/cli.js or wherever your root yargs setup lives)
+// ─────────────────────────────────────────────────────────────────────────────
+
+import { moments } from '../lib/robot-moments.js';
+import { readFileSync } from 'fs';
+
+const { version } = JSON.parse(readFileSync('package.json', 'utf8'));
+
+// Add before yargs .usage() / .epilog():
+moments.greet({ version, name: config.get('user-name') });
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 2. vai ingest  (src/commands/ingest.js)
+// ─────────────────────────────────────────────────────────────────────────────
+
+export async function handler(argv) {
+  // Show thinking animation while scanning files
+  const anim = moments.startThinking(`Scanning ${argv.path}…`);
+
+  try {
+    const files = await scanFiles(argv.path);
+    // Update label mid-animation isn't built-in, so just stop and restart:
+    anim.stop();
+
+    const anim2 = moments.startThinking(`Embedding ${files.length} files…`);
+    const { chunks } = await embedAndIngest(files, argv);
+    anim2.stop('success');
+
+    moments.success(
+      `Indexed ${chunks} chunks from ${files.length} files`,
+      [
+        `Collection: ${argv.collection}`,
+        `Model:      ${argv.model ?? 'voyage-4-large'}`,
+        `Run 'vai search' to test retrieval`,
+      ]
+    );
+  } catch (err) {
+    anim.stop('error');
+    moments.error(err.message, 'Check your MongoDB URI and Voyage AI key');
+    process.exit(1);
+  }
+}
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 3. vai search  (src/commands/search.js)
+// ─────────────────────────────────────────────────────────────────────────────
+
+export async function handler(argv) {
+  const anim = moments.startSearching(
+    `Searching ${argv.collection} · top-${argv.k ?? 5}…`
+  );
+
+  try {
+    const start = Date.now();
+    const hits = await vectorSearch(argv.query, argv);
+    anim.stop(hits.length > 0 ? 'success' : 'idle');
+
+    if (hits.length === 0) {
+      moments.noResults(
+        argv.collection,
+        'Try a broader query or check that documents are ingested'
+      );
+      return;
+    }
+
+    moments.results(hits.length, argv.collection, Date.now() - start);
+    // ... render hits table as normal
+  } catch (err) {
+    anim.stop('error');
+    moments.error(err.message);
+    process.exit(1);
+  }
+}
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 4. vai explain  (src/commands/explain.js)
+// ─────────────────────────────────────────────────────────────────────────────
+
+export async function handler(argv) {
+  moments.explain(argv.topic);
+  // ... existing explain output continues below
+}
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 5. vai init / vai config  (first-run setup)
+// ─────────────────────────────────────────────────────────────────────────────
+
+export async function handler(argv) {
+  moments.setup('Let\'s configure vai for your project…');
+
+  // ... prompts, config writing, etc.
+
+  moments.success(
+    'vai is configured and ready!',
+    ['Run \'vai ingest\' to embed your first documents']
+  );
+}
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 6. vai chat  (src/commands/chat.js) — during RAG retrieval step
+// ─────────────────────────────────────────────────────────────────────────────
+
+async function handleTurn(userMessage, session) {
+  // Show searching animation during retrieval
+  const anim = moments.startSearching(`Searching ${session.collection}…`);
+  const docs = await retrieveAndRerank(userMessage, session);
+  anim.stop();
+
+  // Show thinking animation during LLM generation
+  const anim2 = moments.startThinking(`Generating with ${session.llmModel}…`);
+  const response = await streamGenerate(docs, userMessage, session);
+  anim2.stop();
+
+  // ... stream response to terminal
+}
+
+
+// ─────────────────────────────────────────────────────────────────────────────
+// 7. Plain/pipe mode guard  (add to any command)
+// ─────────────────────────────────────────────────────────────────────────────
+
+// Check if output is being piped — skip robot in that case
+const isInteractive = process.stdout.isTTY && !argv.plain && !argv.json;
+
+if (isInteractive) {
+  moments.greet({ version });
+} else {
+  // plain text output only, no robot
+}