@vespermcp/mcp-server 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Vesper Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,259 @@
+# Vesper MCP Server 🚀
+
+**AI-powered dataset discovery, quality analysis, and preparation** with multimodal support (text, image, audio, video).
+
+Vesper is a Model Context Protocol (MCP) server that helps you find, analyze, and prepare high-quality datasets for machine learning projects. It integrates seamlessly with AI assistants like Claude, providing autonomous dataset workflows.
+
+## ✨ Features
+
+### 🔍 **Dataset Discovery**
+- Search across HuggingFace, Kaggle, UCI ML Repository, and more
+- Intelligent ranking based on quality, safety, and relevance
+- Automatic metadata extraction and enrichment
+
+### 📊 **Quality Analysis**
+- **Text**: Missing data, duplicates, column profiling
+- **Images**: Resolution, corruption, blur detection
+- **Audio**: Sample rate, duration, silence detection
+- **Video**: FPS, frame validation, corruption risk
+- **Unified Reports**: Consolidated quality scores (0-100) with recommendations
+
+### 🛠️ **Data Preparation**
+- Automated cleaning pipelines
+- Format conversion (CSV, JSON, Parquet)
+- Train/test/validation splitting
+- Automatic installation to project directories
+
+### 🎯 **Multimodal Support**
+- Analyze mixed datasets (text + images + audio)
+- Media-specific quality metrics
+- Intelligent modality detection
+
+## 📦 Installation
+
+### Option A: Install via Git (Recommended)
+Install directly from the repository without waiting for npm publishing:
+```bash
+npm install -g git+https://github.com/vesper/mcp-server.git
+```
+
+### Option B: Install Globally from Source
+1. Clone the repository
+2. Run install:
+```bash
+npm install -g .
+```
+
+The postinstall script will automatically:
+- Install Python dependencies (opencv-python, librosa, etc.)
+- Create data directories in `~/.vesper`
+- Display setup instructions
+
+### Manual Python Setup (if needed)
+
+```bash
+pip install opencv-python pillow numpy librosa soundfile
+```
+
+## ⚙️ MCP Configuration
+
+Add Vesper to your MCP settings file:
+
+**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS):
+
+```json
+{
+  "mcpServers": {
+    "vesper": {
+      "command": "node",
+      "args": [
+        "vesper"
+      ],
+      "env": {
+        "KAGGLE_USERNAME": "your-username",
+        "KAGGLE_KEY": "your-api-key",
+        "HF_TOKEN": "your-huggingface-token"
+      }
+    }
+  }
+}
+```
+
+> **Note**: Update the path in `args` if `vesper` is not in your PATH. You can use the full path: `/usr/local/lib/node_modules/vesper-mcp-server/build/index.js` (use `npm root -g` to check location).
+
+### Environment Variables (Optional)
+
+- `KAGGLE_USERNAME` & `KAGGLE_KEY`: For Kaggle dataset access
+- `HF_TOKEN`: For private HuggingFace datasets
+
+## 🚀 Quick Start
+
+After installation and configuration, restart your AI assistant and try:
+
+```
+search_datasets(query="sentiment analysis", limit=5)
+```
+
+```
+prepare_dataset(query="image classification cats vs dogs")
+```
+
+```
+generate_quality_report(
+  dataset_id="huggingface:imdb", 
+  dataset_path="/path/to/data"
+)
+```
+
+## 📚 Available Tools
+
+### Dataset Discovery
+
+#### `search_datasets`
+Search for datasets across multiple sources.
+
+**Parameters:**
+- `query` (string): Search query
+- `limit` (number, optional): Max results (default: 10)
+- `min_quality_score` (number, optional): Minimum quality threshold
+
+**Example:**
+```
+search_datasets(query="medical imaging", limit=5, min_quality_score=70)
+```
+
+---
+
+### Data Preparation
+
+#### `prepare_dataset`
+Download, analyze, and prepare a dataset for use.
+
+**Parameters:**
+- `query` (string): Dataset search query or ID
+
+**Example:**
+```
+prepare_dataset(query="squad")
+```
+
+---
+
+#### `export_dataset`
+Export a prepared dataset to a custom directory with format conversion.
+
+**Parameters:**
+- `dataset_id` (string): Dataset identifier
+- `target_dir` (string): Export directory
+- `format` (string, optional): Output format (csv, json, parquet)
+
+**Example:**
+```
+export_dataset(
+  dataset_id="huggingface:imdb",
+  target_dir="./my-data",
+  format="csv"
+)
+```
+
+---
+
+### Quality Analysis
+
+#### `analyze_image_quality`
+Analyze image datasets for resolution, corruption, and blur.
+
+**Parameters:**
+- `path` (string): Path to image file or folder
+
+**Example:**
+```
+analyze_image_quality(path="/path/to/images")
+```
+
+---
+
+#### `analyze_media_quality`
+Analyze audio/video files for quality metrics.
+
+**Parameters:**
+- `path` (string): Path to media file or folder
+
+**Example:**
+```
+analyze_media_quality(path="/path/to/audio")
+```
+
+---
+
+#### `generate_quality_report`
+Generate a comprehensive unified quality report for multimodal datasets.
+
+**Parameters:**
+- `dataset_id` (string): Dataset identifier
+- `dataset_path` (string): Path to dataset directory
+
+**Example:**
+```
+generate_quality_report(
+  dataset_id="my-dataset",
+  dataset_path="/path/to/data"
+)
+```
+
+---
+
+### Data Splitting
+
+#### `split_dataset`
+Split a dataset into train/test/validation sets.
+
+**Parameters:**
+- `dataset_id` (string): Dataset identifier
+- `train_ratio` (number): Training set ratio (0-1)
+- `test_ratio` (number): Test set ratio (0-1)
+- `val_ratio` (number, optional): Validation set ratio (0-1)
+
+**Example:**
+```
+split_dataset(
+  dataset_id="my-dataset",
+  train_ratio=0.7,
+  test_ratio=0.2,
+  val_ratio=0.1
+)
+```
+
+## 🏗️ Architecture
+
+Vesper is built with:
+- **TypeScript** for the MCP server
+- **Python** for image/audio/video processing
+- **SQLite** for metadata storage
+- **Transformers.js** for semantic search
+
+## 🤝 Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) for details.
+
+## 🐛 Issues & Support
+
+- **Issues**: https://github.com/vesper/mcp-server/issues
+- **Discussions**: https://github.com/vesper/mcp-server/discussions
+
+## 🌟 Acknowledgments
+
+Built with:
+- [Model Context Protocol](https://modelcontextprotocol.io/)
+- [HuggingFace Hub](https://huggingface.co/)
+- [Kaggle API](https://www.kaggle.com/docs/api)
+- [OpenCV](https://opencv.org/)
+- [librosa](https://librosa.org/)
+
+---
+
+Made with ❤️ by the Vesper Team

package/build/cache/cdn.js

@@ -0,0 +1,34 @@
+import fs from "fs";
+import path from "path";
+export class CDNService {
+    baseDir;
+    baseUrl;
+    constructor(baseDir = "data/cdn_mock", baseUrl = "https://cdn.vesper.ai") {
+        this.baseDir = path.resolve(baseDir);
+        this.baseUrl = baseUrl;
+        if (!fs.existsSync(this.baseDir)) {
+            fs.mkdirSync(this.baseDir, { recursive: true });
+        }
+    }
+    /**
+     * Uploads a file to the CDN.
+     * @param fileName Name of the file (including extension)
+     * @param content String or Buffer content
+     * @returns The public URL of the file
+     */
+    async upload(fileName, content) {
+        const filePath = path.join(this.baseDir, fileName);
+        fs.writeFileSync(filePath, content);
+        // Return a simulated cloud URL
+        return `${this.baseUrl}/${fileName}`;
+    }
+    /**
+     * Deletes a file from the CDN.
+     */
+    async delete(fileName) {
+        const filePath = path.join(this.baseDir, fileName);
+        if (fs.existsSync(filePath)) {
+            fs.unlinkSync(filePath);
+        }
+    }
+}

package/build/cache/service.js

@@ -0,0 +1,63 @@
+import crypto from "crypto";
+/**
+ * A simple in-memory cache provider simulating Redis for demonstration.
+ */
+export class MockRedisProvider {
+    store = new Map();
+    async get(key) {
+        const item = this.store.get(key);
+        if (!item)
+            return null;
+        if (item.expiry && Date.now() > item.expiry) {
+            this.store.delete(key);
+            return null;
+        }
+        return item.value;
+    }
+    async set(key, value, ttlSeconds) {
+        const expiry = ttlSeconds ? Date.now() + ttlSeconds * 1000 : null;
+        this.store.set(key, { value, expiry });
+    }
+    async delete(key) {
+        this.store.delete(key);
+    }
+}
+export class CacheService {
+    provider;
+    constructor(provider) {
+        this.provider = provider;
+    }
+    /**
+     * Caches quality reports (TTL: 24h)
+     */
+    async getReport(datasetId) {
+        const key = `report:${datasetId}`;
+        const data = await this.provider.get(key);
+        return data ? JSON.parse(data) : null;
+    }
+    async saveReport(datasetId, report) {
+        const key = `report:${datasetId}`;
+        await this.provider.set(key, JSON.stringify(report), 86400); // 24 hours
+    }
+    /**
+     * Caches cleaning plans by dataset ID and configuration hash
+     */
+    async getPlan(datasetId, config) {
+        const hash = this.generateHash(config);
+        const key = `plan:${datasetId}:${hash}`;
+        const data = await this.provider.get(key);
+        return data ? JSON.parse(data) : null;
+    }
+    async savePlan(datasetId, config, plan) {
+        const hash = this.generateHash(config);
+        const key = `plan:${datasetId}:${hash}`;
+        await this.provider.set(key, JSON.stringify(plan), 3600); // 1 hour
+    }
+    generateHash(obj) {
+        return crypto
+            .createHash("sha256")
+            .update(JSON.stringify(obj))
+            .digest("hex")
+            .substring(0, 16);
+    }
+}

package/build/cleaning/cleaner.js

@@ -0,0 +1,50 @@
+import { spawn } from "child_process";
+import path from "path";
+export class DataCleaner {
+    pythonPath = "python";
+    scriptPath;
+    constructor(projectRoot = process.cwd()) {
+        this.scriptPath = path.join(projectRoot, "src", "python", "cleaner.py");
+    }
+    /**
+     * Execute a list of cleaning operations on a file
+     */
+    async clean(filePath, operations, format) {
+        return new Promise((resolve, reject) => {
+            const args = [
+                this.scriptPath,
+                filePath,
+                JSON.stringify(operations)
+            ];
+            if (format)
+                args.push(format);
+            const process = spawn(this.pythonPath, args);
+            let stdout = "";
+            let stderr = "";
+            process.stdout.on("data", (data) => {
+                stdout += data.toString();
+            });
+            process.stderr.on("data", (data) => {
+                stderr += data.toString();
+            });
+            process.on("close", (code) => {
+                if (code !== 0) {
+                    reject(new Error(`Data Cleaner failed (code ${code}): ${stderr}`));
+                    return;
+                }
+                try {
+                    const result = JSON.parse(stdout);
+                    if (!result.success) {
+                        reject(new Error(result.error));
+                    }
+                    else {
+                        resolve(result);
+                    }
+                }
+                catch (e) {
+                    reject(new Error(`Failed to parse cleaner output: ${stdout}`));
+                }
+            });
+        });
+    }
+}

package/build/cleaning/evaluator.js

@@ -0,0 +1,89 @@
+import * as crypto from "crypto";
+export class RuleEvaluator {
+    /**
+     * Checks if a record matches a rule's condition.
+     */
+    matches(record, condition) {
+        const columnsToTest = condition.column === "*"
+            ? Object.keys(record)
+            : [condition.column];
+        for (const col of columnsToTest) {
+            const val = record[col];
+            if (val === undefined)
+                continue;
+            if (this.testValue(val, condition)) {
+                return true;
+            }
+        }
+        return false;
+    }
+    /**
+     * Applies a rule action to a record.
+     */
+    apply(record, rule) {
+        const newRecord = { ...record };
+        const action = rule.action;
+        const condition = rule.condition;
+        const columnsToApply = condition.column === "*"
+            ? Object.keys(record)
+            : [condition.column];
+        for (const col of columnsToApply) {
+            const val = record[col];
+            if (val === undefined)
+                continue;
+            // Optional: Re-test condition per column if using wildcard
+            if (condition.column === "*" && !this.testValue(val, condition)) {
+                continue;
+            }
+            switch (action.type) {
+                case "Replace":
+                    if (typeof val === "string") {
+                        const regex = new RegExp(action.params.pattern, "g");
+                        newRecord[col] = val.replace(regex, action.params.replacement);
+                    }
+                    break;
+                case "CustomMask":
+                    if (val !== null) {
+                        newRecord[col] = this.maskValue(String(val), action.params);
+                    }
+                    break;
+                case "NormalizeText":
+                    if (typeof val === "string") {
+                        newRecord[col] = action.params.case === "lower"
+                            ? val.toLowerCase()
+                            : val.toUpperCase();
+                    }
+                    break;
+                // Add more handlers as needed...
+            }
+        }
+        return newRecord;
+    }
+    testValue(val, condition) {
+        const { operator, value } = condition;
+        const strVal = String(val);
+        switch (operator) {
+            case "contains":
+                return strVal.includes(String(value));
+            case "equals":
+                return val === value;
+            case "starts_with":
+                return strVal.startsWith(String(value));
+            case "ends_with":
+                return strVal.endsWith(String(value));
+            case "is_null":
+                return (val === null || val === undefined) === value;
+            case "matches_regex":
+                return new RegExp(String(value)).test(strVal);
+            default:
+                return false;
+        }
+    }
+    maskValue(val, params) {
+        if (params.method === "hash") {
+            const salt = params.salt || "";
+            return crypto.createHash("sha256").update(val + salt).digest("hex").substring(0, 12) + "...";
+        }
+        return "MASKED";
+    }
+}

package/build/cleaning/executor.js

@@ -0,0 +1,60 @@
+import { QualityAnalyzer } from "../quality/analyzer.js";
+import { CleaningPlanner } from "./planner.js";
+import { DataCleaner } from "./cleaner.js";
+export class PipelineExecutor {
+    analyzer;
+    planner;
+    cleaner;
+    constructor(projectRoot = process.cwd()) {
+        this.analyzer = new QualityAnalyzer(undefined, projectRoot);
+        this.planner = new CleaningPlanner();
+        this.cleaner = new DataCleaner(projectRoot);
+    }
+    /**
+     * Run the full Auto-Cleaning Pipeline on a dataset file
+     */
+    async runPipeline(datasetId, filePath, outputFormat = "csv", onProgress) {
+        // ... (logging setup)
+        const log = (m) => {
+            console.log(`[Pipeline] ${m}`);
+            if (onProgress)
+                onProgress(m);
+        };
+        log(`Analyzing quality for ${datasetId}...`);
+        const qualityReport = await this.analyzer.analyze(filePath);
+        // 2. Generate Plan
+        log(`Generating cleaning plan...`);
+        const plan = await this.planner.generatePlan(datasetId, qualityReport);
+        // If no cleaning needed, we still might need format conversion
+        const needsConversion = !filePath.toLowerCase().endsWith(`.${outputFormat}`);
+        if (plan.operations.length === 0 && !needsConversion) {
+            log(`No cleaning or conversion needed.`);
+            return {
+                initial_quality: qualityReport,
+                plan,
+                cleaning_result: {
+                    success: true,
+                    rows_affected: 0,
+                    columns_affected: 0,
+                    logs: ["No operations generated."]
+                }
+            };
+        }
+        // 3. Execute Plan (includes conversion if requested)
+        log(`Executing ${plan.operations.length} operations (Format: ${outputFormat})...`);
+        plan.operations.forEach(op => console.log(`  - ${op.type}: ${op.reason}`));
+        const cleaningResult = await this.cleaner.clean(filePath, plan.operations, outputFormat);
+        if (cleaningResult.success) {
+            log(`Cleaning complete. Output: ${cleaningResult.output_path}`);
+        }
+        else {
+            log(`Cleaning failed: ${cleaningResult.error}`);
+        }
+        return {
+            initial_quality: qualityReport,
+            plan,
+            cleaning_result: cleaningResult,
+            final_output_path: cleaningResult.output_path
+        };
+    }
+}

package/build/cleaning/exporter.js

@@ -0,0 +1,87 @@
+export class ScriptGenerator {
+    /**
+     * Generate a standalone Python script for the cleaning plan
+     */
+    generatePythonScript(plan, inputPath) {
+        const timestamp = new Date().toISOString().split('T')[0];
+        let script = `"""
+Vesper Auto-Cleaning Script
+Generated: ${timestamp}
+Dataset ID: ${plan.dataset_id}
+"""
+
+import polars as pl
+import numpy as np
+
+def clean_dataset(file_path):
+    print(f"Loading {file_path}...")
+    
+    # Load Data
+    if file_path.endswith(".csv"):
+        df = pl.read_csv(file_path, ignore_errors=True)
+    elif file_path.endswith(".parquet"):
+        df = pl.read_parquet(file_path)
+    else:
+        raise ValueError("Unsupported format")
+
+    print(f"Initial shape: {df.shape}")
+
+`;
+        // Generate code for each operation
+        plan.operations.forEach((op, index) => {
+            script += `    # Step ${index + 1}: ${op.type}\n`;
+            script += `    # Reason: ${op.reason}\n`;
+            script += this.generateOpCode(op);
+            script += `    print(f"After Step ${index + 1} (${op.type}): {df.shape}")\n\n`;
+        });
+        script += `    # Save Output
+    output_path = file_path.replace(".csv", "_cleaned_repro.csv").replace(".parquet", "_cleaned_repro.parquet")
+    if file_path.endswith(".csv"):
+        df.write_csv(output_path)
+    else:
+        df.write_parquet(output_path)
+        
+    print(f"Done! Saved to {output_path}")
+
+if __name__ == "__main__":
+    # Default input path from generation time, can be overridden
+    INPUT_PATH = r"${inputPath}"
+    clean_dataset(INPUT_PATH)
+`;
+        return script;
+    }
+    generateOpCode(op) {
+        const p = op.params;
+        switch (op.type) {
+            case "RemoveDuplicates":
+                return `    df = df.unique()\n`;
+            case "DropColumns":
+                return `    cols_to_drop = ${JSON.stringify(p.columns)}\n    existing_cols = [c for c in cols_to_drop if c in df.columns]\n    if existing_cols:\n        df = df.drop(existing_cols)\n`;
+            case "FillMissing":
+                if (p.method === "constant") {
+                    const val = typeof p.value === 'string' ? `"${p.value}"` : p.value;
+                    return `    df = df.with_columns(pl.col("${p.column}").fill_null(${val}))\n`;
+                }
+                else if (p.method === "mean") {
+                    return `    mean_val = df["${p.column}"].mean()\n    df = df.with_columns(pl.col("${p.column}").fill_null(mean_val))\n`;
+                }
+                else if (p.method === "median") {
+                    return `    median_val = df["${p.column}"].median()\n    df = df.with_columns(pl.col("${p.column}").fill_null(median_val))\n`;
+                }
+                return `    # Unknown fill method for ${p.column}\n`;
+            case "FixTypes":
+                if (p.type === "float")
+                    return `    df = df.with_columns(pl.col("${p.column}").cast(pl.Float64, strict=False))\n`;
+                if (p.type === "int")
+                    return `    df = df.with_columns(pl.col("${p.column}").cast(pl.Int64, strict=False))\n`;
+                if (p.type === "string")
+                    return `    df = df.with_columns(pl.col("${p.column}").cast(pl.Utf8))\n`;
+                return `    # Unknown type conversion for ${p.column}\n`;
+            case "RemoveOutliers":
+                // IQR implementation inline
+                return `    q1 = df["${p.column}"].quantile(0.25)\n    q3 = df["${p.column}"].quantile(0.75)\n    iqr = q3 - q1\n    lower = q1 - (1.5 * iqr)\n    upper = q3 + (1.5 * iqr)\n    df = df.filter((pl.col("${p.column}") >= lower) & (pl.col("${p.column}") <= upper))\n`;
+            default:
+                return `    # Operation ${op.type} not fully supported in export yet\n`;
+        }
+    }
+}