voyageai-cli 1.20.6 → 1.21.0

package/CHANGELOG.md

@@ -4,49 +4,165 @@ All notable changes to voyageai-cli are documented here.
 
 Format based on [Keep a Changelog](https://keepachangelog.com/).
 
-## [Unreleased]
+## [1.21.0] - 2026-02-10
 
 ### Added
-- `vai ingest` — Bulk import from JSONL/JSON/CSV/text with batching, progress bar, and dry-run
-- `vai similarity` — Compute cosine similarity between texts without MongoDB
-- `vai demo` — Interactive guided walkthrough of all features
+- **`vai generate`** — Emit production code snippets for RAG applications
+  - Components: client, connection, retrieval, ingest, search-api
+  - Targets: Node.js/Express, Next.js/MUI, Python/Flask
+  - Auto-detects target from project files
+  - Reads configuration from `.vai.json`
+- **`vai scaffold`** — Create complete starter projects
+  - Vanilla (9 files), Next.js (13 files), Python (8 files)
+  - Includes server, API routes, client, connection, retrieval, ingest
+  - `.env.example` and README with setup instructions
+- **`vai purge`** — Remove embeddings from MongoDB by criteria
+  - Filter by `--source`, `--before`, `--model`, `--stale`
+  - `--dry-run` for safe preview before deletion
+- **`vai refresh`** — Re-embed documents with new model/settings
+  - `--model` and `--dimensions` for model upgrades
+  - `--rechunk` to re-chunk before re-embedding
+  - Batch processing with progress reporting
+- **`vai eval compare`** — Compare multiple configurations side-by-side
+- **`vai eval --save/--baseline`** — Track quality metrics over time
+- **`vai init`** — Modernized with @clack/prompts and back navigation
+- **Desktop App: Check for Updates** — Menu item in app menu (macOS) / Help menu (Windows/Linux)
+- **Desktop App: Generate tab** — Code generation and scaffold UI
+  - Electron: Create projects on disk with native file dialog
+  - Web: Download as ZIP file
+- **Playground: ZIP scaffold** — `/api/scaffold` endpoint for web mode
+- **Onboarding walkthrough** — Updated to 9 steps covering all tabs
+- **Sidebar reorganized** — Tools (Embed, Compare, Search, Multimodal, Generate) / Learn (Benchmark, Explore, About)
+- 3 new explanation topics: `code-generation`, `scaffolding`, `eval-comparison`
+
+### Changed
+- Template engine (`src/lib/codegen.js`) supports `{{var}}`, `{{#if}}`, `{{#each}}` — no external deps
+- 390 tests (up from 360)
+
+## [1.20.0] - 2026-02-04
+
+### Added
+- **Desktop App** — Signed and notarized macOS DMG via electron-builder
+  - Auto-update via electron-updater
+  - Settings cog in header for API key management
+  - OS keychain encryption for stored credentials
+- **Multimodal tab** — Image ↔ text similarity, cross-modal gallery search
+- **Vector Space Invaders** — Easter egg (Konami code or 7x logo click)
+- Anonymous telemetry to vai.mlynn.org (opt-out available)
+- 4 new multimodal explanation topics
+
+### Changed
+- CLI npm package optimized with `files` whitelist (327KB, was 550MB)
+
+## [1.19.0] - 2026-02-03
+
+### Added
+- **`vai benchmark`** — 8 subcommands for model comparison
+  - `embed`, `rerank`, `similarity`, `cost`, `batch`, `asymmetric`, `quantization`, `space`
+  - Latency (p50/p95), throughput, cost per million tokens
+  - `--save` results to JSON
+- **`vai estimate`** — Cost calculator for symmetric vs asymmetric retrieval
+- Marketing site vai.mlynn.org deployed
+
+## [1.18.0] - 2026-02-02
+
+### Added
+- **`vai eval`** — Evaluate retrieval quality with MRR, nDCG, Recall@K, Precision@K
+  - Supports retrieval and rerank modes
+  - Custom K values and test sets
+- **`vai query`** — Two-stage retrieval (vector search + rerank)
+- **`vai pipeline`** — End-to-end chunk → embed → store
+  - `--create-index` flag for automatic index creation
+  - Progress reporting on stderr
+- **`vai chunk`** — 5 chunking strategies (fixed, sentence, paragraph, recursive, markdown)
+- **`vai init`** — Initialize project with `.vai.json` configuration
+
+## [1.17.0] - 2026-02-01
+
+### Added
+- **Playground** — Web UI at `vai playground`
+  - Embed, Compare, Search, Benchmark tabs
+  - Model selector, dark/light theme toggle
+  - Real-time similarity heatmap
+- **`vai explain`** — 22 interactive concept explainers
+  - Topics: embeddings, reranking, vector-search, RAG, cosine-similarity, etc.
+  - Alias resolution for common terms
+  - Links and "try it" commands for each topic
+- **`vai app`** — Launch Electron desktop app
+- **`vai completions`** — Shell completions for bash and zsh
+- **`vai about`** — Version and system info
+
+## [1.15.0] - 2026-01-30
+
+### Added
+- **voyage-4 model family** support
+  - voyage-4-large, voyage-4, voyage-4-lite
+  - Shared embedding space for asymmetric retrieval
+  - MoE architecture detection
+- **Quantization support** — `--output-dtype` for int8/ubinary embeddings
+- **Dimensions parameter** — `--dimensions` for reduced output dimensions
+
+### Changed
+- Default model updated to voyage-4-lite
+- MODEL_CATALOG expanded with architecture info and benchmark scores
+
+## [1.10.0] - 2026-01-25
+
+### Added
+- **`vai ingest`** — Bulk import from JSONL/JSON/CSV/text
+  - Batching with configurable batch size
+  - Progress bar and dry-run mode
+  - Token estimation and cost preview
+- **`vai similarity`** — Compute cosine similarity without MongoDB
+- **`vai demo`** — Interactive guided walkthrough
 - ASCII banner when running `vai` with no arguments
-- CONTRIBUTING.md for open-source contributors
-- This changelog
 
-## [1.1.0] - 2026-02-03
+### Changed
+- Improved error messages with actionable hints
+- Rate limit retry with exponential backoff (up to 3 attempts)
+
+## [1.5.0] - 2026-01-20
+
+### Added
+- **PDF support** — Optional pdf-parse dependency for `vai pipeline`
+- **HTML stripping** — Automatic for .html files
+- **Readers module** — Unified file reading with type detection
+- **Directory scanning** — Recursive with extension filtering
+
+### Fixed
+- JSONL parsing edge cases
+- Empty chunk handling
+
+## [1.1.0] - 2026-01-15
 
 ### Added
-- `vai config` — Persistent config management (`~/.vai/config.json`)
+- **`vai config`** — Persistent config management (`~/.vai/config.json`)
   - `set`, `get`, `delete`, `path`, `reset` subcommands
   - Secrets masked in output, config file chmod 600
-  - `--stdin` flag for secure key input (avoids shell history)
-- `vai ping` — Test API and MongoDB connectivity
+  - `--stdin` flag for secure key input
+- **`vai ping`** — Test API and MongoDB connectivity
 - `.env` file support via dotenv
-- Colored output with picocolors (green ✓, red ✗, score-based colors)
-- Animated spinners on all network operations
-- npm update notifier (checks daily, non-blocking)
+- Colored output with picocolors
+- Animated spinners on network operations
+- npm update notifier (daily, non-blocking)
 - GitHub Actions CI (Node 18, 20, 22)
-- README badges (CI, npm, license, node version)
-- Credential priority chain: env var → .env → config file
-- Security documentation in README
 
 ### Fixed
-- `ping` command now falls back to config file for API key and MongoDB URI
-- `rerank` endpoint corrected from `/v1/reranking` to `/v1/rerank`
-- `index create` parseInt handling for dimensions (was producing NaN)
+- `rerank` endpoint corrected to `/v1/rerank`
+- `index create` parseInt handling for dimensions
 
-## [1.0.0] - 2026-02-03
+## [1.0.0] - 2026-01-10
 
 ### Added
-- `vai embed` — Generate embeddings (text, file, stdin, bulk)
-- `vai rerank` — Rerank documents with relevance scoring
-- `vai store` — Embed and insert into MongoDB Atlas (single + batch JSONL)
-- `vai search` — $vectorSearch with pre-filter support
-- `vai index` — Create, list, delete Atlas Vector Search indexes
-- `vai models` — List available Voyage AI models with pricing
+- **`vai embed`** — Generate embeddings (text, file, stdin, bulk)
+- **`vai rerank`** — Rerank documents with relevance scoring
+- **`vai store`** — Embed and insert into MongoDB Atlas
+- **`vai search`** — $vectorSearch with pre-filter support
+- **`vai index`** — Create, list, delete Atlas Vector Search indexes
+- **`vai models`** — List available Voyage AI models with pricing
 - REST API integration with `https://ai.mongodb.com/v1/`
 - MongoDB Atlas Vector Search integration
 - API retry on 429 with exponential backoff
 - `--json` and `--quiet` flags on all commands
 - 50+ unit tests
+- MIT license

package/README.md

@@ -196,6 +196,91 @@ Creates `.vai.json` with your defaults — model, database, collection, chunking
 }
 ```
 
+### Code Generation & Scaffolding
+
+#### `vai generate` — Production code snippets
+
+Generate ready-to-use code from your `.vai.json` config:
+
+```bash
+# List available components
+vai generate --list
+
+# Generate and pipe to files
+vai generate client > lib/voyage.js
+vai generate retrieval > lib/retrieval.js
+vai generate search-api > routes/search.js
+
+# Different targets
+vai generate client --target python    # Flask
+vai generate retrieval --target nextjs # Next.js + MUI
+```
+
+Components: `client`, `connection`, `retrieval`, `ingest`, `search-api`
+
+Targets: `vanilla` (Node.js/Express), `nextjs` (Next.js + MUI), `python` (Flask)
+
+#### `vai scaffold` — Complete starter projects
+
+Create a full project directory with all files pre-configured:
+
+```bash
+# Node.js + Express API (9 files)
+vai scaffold my-rag-api
+
+# Next.js + Material UI (13 files)
+vai scaffold my-app --target nextjs
+
+# Python + Flask (8 files)
+vai scaffold flask-api --target python
+
+# Preview without creating files
+vai scaffold my-app --dry-run
+```
+
+Each project includes: server, API routes, Voyage AI client, MongoDB connection, retrieval module, ingestion pipeline, `.env.example`, and README.
+
+### Data Lifecycle
+
+#### `vai purge` — Remove stale embeddings
+
+Remove embeddings from MongoDB based on criteria:
+
+```bash
+# Remove docs embedded with an old model
+vai purge --model voyage-3.5
+
+# Remove docs whose source files no longer exist
+vai purge --stale
+
+# Remove docs older than a date
+vai purge --before 2026-01-01
+
+# Filter by source pattern
+vai purge --source "docs/old/*.md"
+
+# Preview before deleting
+vai purge --model voyage-3.5 --dry-run
+```
+
+#### `vai refresh` — Re-embed with new settings
+
+Re-embed documents in-place with a new model, dimensions, or chunk settings:
+
+```bash
+# Upgrade to a new model
+vai refresh --model voyage-4-large
+
+# Change dimensions for cost savings
+vai refresh --model voyage-4-large --dimensions 256
+
+# Re-chunk with a better strategy, then re-embed
+vai refresh --rechunk --strategy markdown --chunk-size 1024
+
+# Preview what would change
+vai refresh --model voyage-4-large --dry-run
+```
+
 ### Core Workflow
 
 #### `vai pipeline` — Chunk → embed → store
@@ -345,6 +430,37 @@ vai benchmark quantization --model voyage-4-large --dtypes float,int8,ubinary
 vai benchmark cost --tokens 500 --volumes 100,1000,10000,100000
 ```
 
+### Evaluation
+
+Measure and compare your retrieval quality:
+
+```bash
+# Evaluate retrieval pipeline
+vai eval --test-set test.jsonl --db myapp --collection docs
+
+# Save results for later comparison
+vai eval --test-set test.jsonl --save baseline.json
+
+# Compare against a baseline (shows deltas)
+vai eval --test-set test.jsonl --baseline baseline.json
+
+# Compare multiple configurations
+vai eval compare --test-set test.jsonl --configs baseline.json,experiment.json
+
+# Evaluate reranking in isolation
+vai eval --mode rerank --test-set rerank-test.jsonl
+
+# Compare rerank models
+vai eval --mode rerank --models "rerank-2.5,rerank-2.5-lite" --test-set test.jsonl
+```
+
+**Metrics:** MRR, nDCG@K, Recall@K, MAP, Precision@K
+
+**Test set format (JSONL):**
+```json
+{"query": "What is vector search?", "relevant": ["doc_id_1", "doc_id_2"]}
+```
+
 ### Learn
 
 Interactive explanations of key concepts:
@@ -394,21 +510,33 @@ Covers all 22 commands, subcommands, flags, model names, and explain topics.
 
 | Command | Description |
 |---------|-------------|
+| **Project Setup** | |
 | `vai init` | Initialize project with `.vai.json` |
+| `vai generate` | Generate code snippets (retrieval, ingest, client) |
+| `vai scaffold` | Create complete starter projects |
+| **RAG Pipeline** | |
 | `vai pipeline` | Chunk → embed → store (end-to-end) |
 | `vai query` | Search + rerank (two-stage retrieval) |
 | `vai chunk` | Chunk documents (5 strategies) |
 | `vai estimate` | Cost estimator (symmetric vs asymmetric) |
+| **Embeddings** | |
 | `vai embed` | Generate embeddings |
 | `vai rerank` | Rerank documents by relevance |
 | `vai similarity` | Compare text similarity |
+| **Data Management** | |
 | `vai store` | Embed and store single documents |
 | `vai ingest` | Bulk import with progress |
 | `vai search` | Vector similarity search |
 | `vai index` | Manage Atlas Vector Search indexes |
-| `vai models` | List models, benchmarks, architecture |
+| `vai purge` | Remove embeddings by criteria |
+| `vai refresh` | Re-embed with new model/settings |
+| **Evaluation** | |
+| `vai eval` | Evaluate retrieval quality (MRR, nDCG, Recall) |
+| `vai eval compare` | Compare configurations side-by-side |
 | `vai benchmark` | 8 subcommands for model comparison |
-| `vai explain` | 17 interactive concept explainers |
+| **Tools & Learning** | |
+| `vai models` | List models, benchmarks, architecture |
+| `vai explain` | 25 interactive concept explainers |
 | `vai config` | Manage persistent configuration |
 | `vai ping` | Test API and MongoDB connectivity |
 | `vai playground` | Interactive web playground |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "voyageai-cli",
-  "version": "1.20.6",
+  "version": "1.21.0",
   "description": "CLI for Voyage AI embeddings, reranking, and MongoDB Atlas Vector Search",
   "bin": {
     "vai": "./src/cli.js"

package/src/cli.js

@@ -26,6 +26,10 @@ const { registerChunk } = require('./commands/chunk');
 const { registerQuery } = require('./commands/query');
 const { registerPipeline } = require('./commands/pipeline');
 const { registerEval } = require('./commands/eval');
+const { registerGenerate } = require('./commands/generate');
+const { registerScaffold } = require('./commands/scaffold');
+const { register: registerPurge } = require('./commands/purge');
+const { register: registerRefresh } = require('./commands/refresh');
 const { registerApp } = require('./commands/app');
 const { registerAbout } = require('./commands/about');
 const { register: registerDoctor } = require('./commands/doctor');
@@ -60,6 +64,10 @@ registerChunk(program);
 registerQuery(program);
 registerPipeline(program);
 registerEval(program);
+registerGenerate(program);
+registerScaffold(program);
+registerPurge(program);
+registerRefresh(program);
 registerApp(program);
 registerAbout(program);
 registerDoctor(program);