voyageai-cli 1.20.6 → 1.22.0

package/CHANGELOG.md

@@ -4,49 +4,165 @@ All notable changes to voyageai-cli are documented here.
 
 Format based on [Keep a Changelog](https://keepachangelog.com/).
 
-## [Unreleased]
+## [1.21.0] - 2026-02-10
 
 ### Added
-- `vai ingest` — Bulk import from JSONL/JSON/CSV/text with batching, progress bar, and dry-run
-- `vai similarity` — Compute cosine similarity between texts without MongoDB
-- `vai demo` — Interactive guided walkthrough of all features
+- **`vai generate`** — Emit production code snippets for RAG applications
+  - Components: client, connection, retrieval, ingest, search-api
+  - Targets: Node.js/Express, Next.js/MUI, Python/Flask
+  - Auto-detects target from project files
+  - Reads configuration from `.vai.json`
+- **`vai scaffold`** — Create complete starter projects
+  - Vanilla (9 files), Next.js (13 files), Python (8 files)
+  - Includes server, API routes, client, connection, retrieval, ingest
+  - `.env.example` and README with setup instructions
+- **`vai purge`** — Remove embeddings from MongoDB by criteria
+  - Filter by `--source`, `--before`, `--model`, `--stale`
+  - `--dry-run` for safe preview before deletion
+- **`vai refresh`** — Re-embed documents with new model/settings
+  - `--model` and `--dimensions` for model upgrades
+  - `--rechunk` to re-chunk before re-embedding
+  - Batch processing with progress reporting
+- **`vai eval compare`** — Compare multiple configurations side-by-side
+- **`vai eval --save/--baseline`** — Track quality metrics over time
+- **`vai init`** — Modernized with @clack/prompts and back navigation
+- **Desktop App: Check for Updates** — Menu item in app menu (macOS) / Help menu (Windows/Linux)
+- **Desktop App: Generate tab** — Code generation and scaffold UI
+  - Electron: Create projects on disk with native file dialog
+  - Web: Download as ZIP file
+- **Playground: ZIP scaffold** — `/api/scaffold` endpoint for web mode
+- **Onboarding walkthrough** — Updated to 9 steps covering all tabs
+- **Sidebar reorganized** — Tools (Embed, Compare, Search, Multimodal, Generate) / Learn (Benchmark, Explore, About)
+- 3 new explanation topics: `code-generation`, `scaffolding`, `eval-comparison`
+
+### Changed
+- Template engine (`src/lib/codegen.js`) supports `{{var}}`, `{{#if}}`, `{{#each}}` — no external deps
+- 390 tests (up from 360)
+
+## [1.20.0] - 2026-02-04
+
+### Added
+- **Desktop App** — Signed and notarized macOS DMG via electron-builder
+  - Auto-update via electron-updater
+  - Settings cog in header for API key management
+  - OS keychain encryption for stored credentials
+- **Multimodal tab** — Image ↔ text similarity, cross-modal gallery search
+- **Vector Space Invaders** — Easter egg (Konami code or 7x logo click)
+- Anonymous telemetry to vai.mlynn.org (opt-out available)
+- 4 new multimodal explanation topics
+
+### Changed
+- CLI npm package optimized with `files` whitelist (327KB, was 550MB)
+
+## [1.19.0] - 2026-02-03
+
+### Added
+- **`vai benchmark`** — 8 subcommands for model comparison
+  - `embed`, `rerank`, `similarity`, `cost`, `batch`, `asymmetric`, `quantization`, `space`
+  - Latency (p50/p95), throughput, cost per million tokens
+  - `--save` results to JSON
+- **`vai estimate`** — Cost calculator for symmetric vs asymmetric retrieval
+- Marketing site vai.mlynn.org deployed
+
+## [1.18.0] - 2026-02-02
+
+### Added
+- **`vai eval`** — Evaluate retrieval quality with MRR, nDCG, Recall@K, Precision@K
+  - Supports retrieval and rerank modes
+  - Custom K values and test sets
+- **`vai query`** — Two-stage retrieval (vector search + rerank)
+- **`vai pipeline`** — End-to-end chunk → embed → store
+  - `--create-index` flag for automatic index creation
+  - Progress reporting on stderr
+- **`vai chunk`** — 5 chunking strategies (fixed, sentence, paragraph, recursive, markdown)
+- **`vai init`** — Initialize project with `.vai.json` configuration
+
+## [1.17.0] - 2026-02-01
+
+### Added
+- **Playground** — Web UI at `vai playground`
+  - Embed, Compare, Search, Benchmark tabs
+  - Model selector, dark/light theme toggle
+  - Real-time similarity heatmap
+- **`vai explain`** — 22 interactive concept explainers
+  - Topics: embeddings, reranking, vector-search, RAG, cosine-similarity, etc.
+  - Alias resolution for common terms
+  - Links and "try it" commands for each topic
+- **`vai app`** — Launch Electron desktop app
+- **`vai completions`** — Shell completions for bash and zsh
+- **`vai about`** — Version and system info
+
+## [1.15.0] - 2026-01-30
+
+### Added
+- **voyage-4 model family** support
+  - voyage-4-large, voyage-4, voyage-4-lite
+  - Shared embedding space for asymmetric retrieval
+  - MoE architecture detection
+- **Quantization support** — `--output-dtype` for int8/ubinary embeddings
+- **Dimensions parameter** — `--dimensions` for reduced output dimensions
+
+### Changed
+- Default model updated to voyage-4-lite
+- MODEL_CATALOG expanded with architecture info and benchmark scores
+
+## [1.10.0] - 2026-01-25
+
+### Added
+- **`vai ingest`** — Bulk import from JSONL/JSON/CSV/text
+  - Batching with configurable batch size
+  - Progress bar and dry-run mode
+  - Token estimation and cost preview
+- **`vai similarity`** — Compute cosine similarity without MongoDB
+- **`vai demo`** — Interactive guided walkthrough
 - ASCII banner when running `vai` with no arguments
-- CONTRIBUTING.md for open-source contributors
-- This changelog
 
-## [1.1.0] - 2026-02-03
+### Changed
+- Improved error messages with actionable hints
+- Rate limit retry with exponential backoff (up to 3 attempts)
+
+## [1.5.0] - 2026-01-20
+
+### Added
+- **PDF support** — Optional pdf-parse dependency for `vai pipeline`
+- **HTML stripping** — Automatic for .html files
+- **Readers module** — Unified file reading with type detection
+- **Directory scanning** — Recursive with extension filtering
+
+### Fixed
+- JSONL parsing edge cases
+- Empty chunk handling
+
+## [1.1.0] - 2026-01-15
 
 ### Added
-- `vai config` — Persistent config management (`~/.vai/config.json`)
+- **`vai config`** — Persistent config management (`~/.vai/config.json`)
   - `set`, `get`, `delete`, `path`, `reset` subcommands
   - Secrets masked in output, config file chmod 600
-  - `--stdin` flag for secure key input (avoids shell history)
-- `vai ping` — Test API and MongoDB connectivity
+  - `--stdin` flag for secure key input
+- **`vai ping`** — Test API and MongoDB connectivity
 - `.env` file support via dotenv
-- Colored output with picocolors (green ✓, red ✗, score-based colors)
-- Animated spinners on all network operations
-- npm update notifier (checks daily, non-blocking)
+- Colored output with picocolors
+- Animated spinners on network operations
+- npm update notifier (daily, non-blocking)
 - GitHub Actions CI (Node 18, 20, 22)
-- README badges (CI, npm, license, node version)
-- Credential priority chain: env var → .env → config file
-- Security documentation in README
 
 ### Fixed
-- `ping` command now falls back to config file for API key and MongoDB URI
-- `rerank` endpoint corrected from `/v1/reranking` to `/v1/rerank`
-- `index create` parseInt handling for dimensions (was producing NaN)
+- `rerank` endpoint corrected to `/v1/rerank`
+- `index create` parseInt handling for dimensions
 
-## [1.0.0] - 2026-02-03
+## [1.0.0] - 2026-01-10
 
 ### Added
-- `vai embed` — Generate embeddings (text, file, stdin, bulk)
-- `vai rerank` — Rerank documents with relevance scoring
-- `vai store` — Embed and insert into MongoDB Atlas (single + batch JSONL)
-- `vai search` — $vectorSearch with pre-filter support
-- `vai index` — Create, list, delete Atlas Vector Search indexes
-- `vai models` — List available Voyage AI models with pricing
+- **`vai embed`** — Generate embeddings (text, file, stdin, bulk)
+- **`vai rerank`** — Rerank documents with relevance scoring
+- **`vai store`** — Embed and insert into MongoDB Atlas
+- **`vai search`** — $vectorSearch with pre-filter support
+- **`vai index`** — Create, list, delete Atlas Vector Search indexes
+- **`vai models`** — List available Voyage AI models with pricing
 - REST API integration with `https://ai.mongodb.com/v1/`
 - MongoDB Atlas Vector Search integration
 - API retry on 429 with exponential backoff
 - `--json` and `--quiet` flags on all commands
 - 50+ unit tests
+- MIT license

package/README.md

@@ -196,6 +196,91 @@ Creates `.vai.json` with your defaults — model, database, collection, chunking
 }
 ```
 
+### Code Generation & Scaffolding
+
+#### `vai generate` — Production code snippets
+
+Generate ready-to-use code from your `.vai.json` config:
+
+```bash
+# List available components
+vai generate --list
+
+# Generate and pipe to files
+vai generate client > lib/voyage.js
+vai generate retrieval > lib/retrieval.js
+vai generate search-api > routes/search.js
+
+# Different targets
+vai generate client --target python    # Flask
+vai generate retrieval --target nextjs # Next.js + MUI
+```
+
+Components: `client`, `connection`, `retrieval`, `ingest`, `search-api`
+
+Targets: `vanilla` (Node.js/Express), `nextjs` (Next.js + MUI), `python` (Flask)
+
+#### `vai scaffold` — Complete starter projects
+
+Create a full project directory with all files pre-configured:
+
+```bash
+# Node.js + Express API (9 files)
+vai scaffold my-rag-api
+
+# Next.js + Material UI (13 files)
+vai scaffold my-app --target nextjs
+
+# Python + Flask (8 files)
+vai scaffold flask-api --target python
+
+# Preview without creating files
+vai scaffold my-app --dry-run
+```
+
+Each project includes: server, API routes, Voyage AI client, MongoDB connection, retrieval module, ingestion pipeline, `.env.example`, and README.
+
+### Data Lifecycle
+
+#### `vai purge` — Remove stale embeddings
+
+Remove embeddings from MongoDB based on criteria:
+
+```bash
+# Remove docs embedded with an old model
+vai purge --model voyage-3.5
+
+# Remove docs whose source files no longer exist
+vai purge --stale
+
+# Remove docs older than a date
+vai purge --before 2026-01-01
+
+# Filter by source pattern
+vai purge --source "docs/old/*.md"
+
+# Preview before deleting
+vai purge --model voyage-3.5 --dry-run
+```
+
+#### `vai refresh` — Re-embed with new settings
+
+Re-embed documents in-place with a new model, dimensions, or chunk settings:
+
+```bash
+# Upgrade to a new model
+vai refresh --model voyage-4-large
+
+# Change dimensions for cost savings
+vai refresh --model voyage-4-large --dimensions 256
+
+# Re-chunk with a better strategy, then re-embed
+vai refresh --rechunk --strategy markdown --chunk-size 1024
+
+# Preview what would change
+vai refresh --model voyage-4-large --dry-run
+```
+
 ### Core Workflow
 
 #### `vai pipeline` — Chunk → embed → store
@@ -345,6 +430,37 @@ vai benchmark quantization --model voyage-4-large --dtypes float,int8,ubinary
 vai benchmark cost --tokens 500 --volumes 100,1000,10000,100000
 ```
 
+### Evaluation
+
+Measure and compare your retrieval quality:
+
+```bash
+# Evaluate retrieval pipeline
+vai eval --test-set test.jsonl --db myapp --collection docs
+
+# Save results for later comparison
+vai eval --test-set test.jsonl --save baseline.json
+
+# Compare against a baseline (shows deltas)
+vai eval --test-set test.jsonl --baseline baseline.json
+
+# Compare multiple configurations
+vai eval compare --test-set test.jsonl --configs baseline.json,experiment.json
+
+# Evaluate reranking in isolation
+vai eval --mode rerank --test-set rerank-test.jsonl
+
+# Compare rerank models
+vai eval --mode rerank --models "rerank-2.5,rerank-2.5-lite" --test-set test.jsonl
+```
+
+**Metrics:** MRR, nDCG@K, Recall@K, MAP, Precision@K
+
+**Test set format (JSONL):**
+```json
+{"query": "What is vector search?", "relevant": ["doc_id_1", "doc_id_2"]}
+```
+
 ### Learn
 
 Interactive explanations of key concepts:
@@ -394,21 +510,33 @@ Covers all 22 commands, subcommands, flags, model names, and explain topics.
 
 | Command | Description |
 |---------|-------------|
+| **Project Setup** | |
 | `vai init` | Initialize project with `.vai.json` |
+| `vai generate` | Generate code snippets (retrieval, ingest, client) |
+| `vai scaffold` | Create complete starter projects |
+| **RAG Pipeline** | |
 | `vai pipeline` | Chunk → embed → store (end-to-end) |
 | `vai query` | Search + rerank (two-stage retrieval) |
 | `vai chunk` | Chunk documents (5 strategies) |
 | `vai estimate` | Cost estimator (symmetric vs asymmetric) |
+| **Embeddings** | |
 | `vai embed` | Generate embeddings |
 | `vai rerank` | Rerank documents by relevance |
 | `vai similarity` | Compare text similarity |
+| **Data Management** | |
 | `vai store` | Embed and store single documents |
 | `vai ingest` | Bulk import with progress |
 | `vai search` | Vector similarity search |
 | `vai index` | Manage Atlas Vector Search indexes |
-| `vai models` | List models, benchmarks, architecture |
+| `vai purge` | Remove embeddings by criteria |
+| `vai refresh` | Re-embed with new model/settings |
+| **Evaluation** | |
+| `vai eval` | Evaluate retrieval quality (MRR, nDCG, Recall) |
+| `vai eval compare` | Compare configurations side-by-side |
 | `vai benchmark` | 8 subcommands for model comparison |
-| `vai explain` | 17 interactive concept explainers |
+| **Tools & Learning** | |
+| `vai models` | List models, benchmarks, architecture |
+| `vai explain` | 25 interactive concept explainers |
 | `vai config` | Manage persistent configuration |
 | `vai ping` | Test API and MongoDB connectivity |
 | `vai playground` | Interactive web playground |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "voyageai-cli",
-  "version": "1.20.6",
+  "version": "1.22.0",
   "description": "CLI for Voyage AI embeddings, reranking, and MongoDB Atlas Vector Search",
   "bin": {
     "vai": "./src/cli.js"
@@ -37,9 +37,10 @@
     "test": "node --test test/**/*.test.js"
   },
   "engines": {
-    "node": ">=18.0.0"
+    "node": ">=20.0.0"
   },
   "dependencies": {
+    "@clack/prompts": "^1.0.0",
     "commander": "^12.0.0",
     "dotenv": "^17.2.3",
     "mongodb": "^6.0.0",

package/src/cli.js

@@ -26,10 +26,15 @@ const { registerChunk } = require('./commands/chunk');
 const { registerQuery } = require('./commands/query');
 const { registerPipeline } = require('./commands/pipeline');
 const { registerEval } = require('./commands/eval');
+const { registerGenerate } = require('./commands/generate');
+const { registerScaffold } = require('./commands/scaffold');
+const { register: registerPurge } = require('./commands/purge');
+const { register: registerRefresh } = require('./commands/refresh');
 const { registerApp } = require('./commands/app');
 const { registerAbout } = require('./commands/about');
 const { register: registerDoctor } = require('./commands/doctor');
 const { register: registerQuickstart } = require('./commands/quickstart');
+const { registerBug } = require('./commands/bug');
 const { showBanner, showQuickStart, getVersion } = require('./lib/banner');
 
 const version = getVersion();
@@ -60,10 +65,15 @@ registerChunk(program);
 registerQuery(program);
 registerPipeline(program);
 registerEval(program);
+registerGenerate(program);
+registerScaffold(program);
+registerPurge(program);
+registerRefresh(program);
 registerApp(program);
 registerAbout(program);
 registerDoctor(program);
 registerQuickstart(program);
+registerBug(program);
 
 // Append disclaimer to all help output
 program.addHelpText('after', `

package/src/commands/bug.js

@@ -0,0 +1,249 @@
+'use strict';
+
+const os = require('os');
+const pc = require('picocolors');
+const ui = require('../lib/ui');
+const { send: sendTelemetry } = require('../lib/telemetry');
+
+// Try to get package version safely
+function getVersion() {
+  try {
+    return require('../../package.json').version;
+  } catch {
+    return 'unknown';
+  }
+}
+
+const GITHUB_ISSUES_URL = 'https://github.com/mrlynn/voyageai-cli/issues/new';
+const BUG_API_URL = 'https://vai.mlynn.org/api/bugs';
+
+/**
+ * Generate a GitHub issue URL with pre-filled template
+ */
+function generateGitHubUrl(title, description, context = {}) {
+  const issueTitle = encodeURIComponent(`[Bug] ${title || 'Bug Report'}`);
+  
+  const body = `## Description
+${description || 'Describe the bug here...'}
+
+## Steps to Reproduce
+1. 
+2. 
+3. 
+
+## Expected Behavior
+
+
+## Actual Behavior
+
+
+## Environment
+- **CLI Version:** ${context.cliVersion || getVersion()}
+- **Node Version:** ${process.version}
+- **Platform:** ${os.platform()} ${os.release()}
+- **Arch:** ${os.arch()}
+${context.command ? `- **Command:** \`${context.command}\`` : ''}
+
+## Additional Context
+${context.errorMessage ? `### Error\n\`\`\`\n${context.errorMessage}\n\`\`\`` : 'Add any other context here.'}
+`;
+
+  return `${GITHUB_ISSUES_URL}?title=${issueTitle}&body=${encodeURIComponent(body)}&labels=bug`;
+}
+
+/**
+ * Submit bug report to API
+ */
+async function submitBugReport(data) {
+  try {
+    const response = await fetch(BUG_API_URL, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({
+        ...data,
+        source: 'cli',
+        cliVersion: getVersion(),
+        platform: os.platform(),
+        arch: os.arch(),
+        nodeVersion: process.version,
+      }),
+    });
+
+    if (!response.ok) {
+      const error = await response.json().catch(() => ({}));
+      throw new Error(error.error || `HTTP ${response.status}`);
+    }
+
+    return await response.json();
+  } catch (error) {
+    throw new Error(`Failed to submit bug report: ${error.message}`);
+  }
+}
+
+/**
+ * Open URL in default browser
+ */
+function openUrl(url) {
+  const { exec } = require('child_process');
+  const command = os.platform() === 'darwin' ? 'open' :
+                  os.platform() === 'win32' ? 'start' : 'xdg-open';
+  exec(`${command} "${url}"`);
+}
+
+/**
+ * Interactive bug report (when no arguments provided)
+ */
+async function interactiveBugReport() {
+  const readline = require('readline');
+  const rl = readline.createInterface({
+    input: process.stdin,
+    output: process.stdout,
+  });
+
+  const question = (prompt) => new Promise((resolve) => {
+    rl.question(prompt, resolve);
+  });
+
+  console.log(ui.info('🐛 Bug Reporter'));
+  console.log(ui.dim('Report issues with the Vai CLI\n'));
+
+  try {
+    const title = await question(ui.label('Title', 'Brief description of the bug') + '\n> ');
+    if (!title.trim()) {
+      console.log(ui.warn('Bug report cancelled.'));
+      rl.close();
+      return;
+    }
+
+    const description = await question(ui.label('Description', 'What happened?') + '\n> ');
+    const steps = await question(ui.label('Steps to Reproduce', 'Optional, press Enter to skip') + '\n> ');
+    const email = await question(ui.label('Email', 'Optional, for follow-up') + '\n> ');
+
+    console.log('');
+    const method = await question('Submit to:\n  [1] Bug tracker (anonymous)\n  [2] GitHub Issues (public)\n  [3] Both\n> ');
+
+    rl.close();
+
+    const bugData = {
+      title: title.trim(),
+      description: description.trim(),
+      stepsToReproduce: steps.trim() || null,
+      email: email.trim() || null,
+    };
+
+    if (method === '1' || method === '3') {
+      console.log(ui.dim('\nSubmitting to bug tracker...'));
+      try {
+        const result = await submitBugReport(bugData);
+        console.log(ui.success(`Bug submitted! ID: ${result.bugId}`));
+      } catch (error) {
+        console.log(ui.error(error.message));
+      }
+    }
+
+    if (method === '2' || method === '3') {
+      const url = generateGitHubUrl(bugData.title, bugData.description, {
+        cliVersion: getVersion(),
+      });
+      console.log(ui.dim('\nOpening GitHub...'));
+      openUrl(url);
+      console.log(ui.success('GitHub issue page opened in browser'));
+    }
+
+    if (!['1', '2', '3'].includes(method)) {
+      console.log(ui.warn('No submission method selected.'));
+    }
+
+  } catch (error) {
+    rl.close();
+    console.error(ui.error(`Error: ${error.message}`));
+  }
+}
+
+/**
+ * Main bug command
+ */
+async function bugCommand(args, flags) {
+  // --github flag: open GitHub issues directly
+  if (flags.github || flags.g) {
+    const title = args.join(' ');
+    const url = generateGitHubUrl(title, '', { cliVersion: getVersion() });
+    console.log(ui.info('Opening GitHub Issues...'));
+    openUrl(url);
+    return;
+  }
+
+  // --quick flag: quick submit with just title
+  if (flags.quick || flags.q) {
+    const title = args.join(' ');
+    if (!title) {
+      console.error(ui.error('Please provide a bug title: vai bug --quick "Something broke"'));
+      process.exit(1);
+    }
+
+    console.log(ui.dim('Submitting quick bug report...'));
+    try {
+      const result = await submitBugReport({
+        title,
+        description: title,
+      });
+      console.log(ui.success(`Bug submitted! ID: ${result.bugId}`));
+      console.log(ui.dim(`Create GitHub issue: ${result.githubIssueUrl}`));
+    } catch (error) {
+      console.error(ui.error(error.message));
+      process.exit(1);
+    }
+    return;
+  }
+
+  // If title provided as argument, use quick mode
+  if (args.length > 0) {
+    const title = args.join(' ');
+    console.log(ui.dim('Submitting bug report...'));
+    try {
+      const result = await submitBugReport({
+        title,
+        description: title,
+      });
+      console.log(ui.success(`Bug submitted! ID: ${result.bugId}`));
+      console.log(ui.dim('To create a GitHub issue with more details:'));
+      console.log(ui.dim(`  ${result.githubIssueUrl.slice(0, 80)}...`));
+    } catch (error) {
+      console.error(ui.error(error.message));
+      process.exit(1);
+    }
+    return;
+  }
+
+  // No arguments: interactive mode
+  await interactiveBugReport();
+}
+
+/**
+ * Register the bug command with Commander
+ */
+function registerBug(program) {
+  program
+    .command('bug [title...]')
+    .description('Report a bug or issue with the Vai CLI')
+    .option('-g, --github', 'Open GitHub Issues in browser')
+    .option('-q, --quick', 'Quick submit (title only, no interaction)')
+    .action(async (titleParts, options) => {
+      sendTelemetry('bug', { 
+        method: options.github ? 'github' : options.quick ? 'quick' : 'interactive' 
+      });
+      
+      const args = titleParts || [];
+      const flags = {
+        github: options.github,
+        g: options.github,
+        quick: options.quick,
+        q: options.quick,
+      };
+      
+      await bugCommand(args, flags);
+    });
+}
+
+module.exports = { registerBug };
+module.exports.bugCommand = bugCommand;