vaspera 2.9.0 → 2.9.2

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 2.9.2
+
+### Patch Changes
+
+- [#30](https://github.com/RCOLKITT/hardening-mcp/pull/30) [`8110af7`](https://github.com/RCOLKITT/hardening-mcp/commit/8110af76da720332e43f296b7357987e7edec533) Thanks [@RCOLKITT](https://github.com/RCOLKITT)! - ## Telemetry Integration
+
+  - Wired up telemetry tracking to certification tools (`certification_scan`, `agent_cert_scan`, `certification_finalize`)
+  - Added scan registry for persistent analytics storage
+  - Telemetry is opt-in via `VASPERA_TELEMETRY_ENABLED` environment variable
+  - Privacy-respecting: repo URL, org name, and email require explicit opt-in
+  - Backend API endpoint for receiving telemetry events with rate limiting
+
 All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
@@ -12,6 +24,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 #### Optimization Plan Modules
 
 ##### Corpus Expansion (P0)
+
 - 7 new payload categories bringing total from 220 to 430+ payloads
 - `multi-turn.json` - 30 payloads for context-building attacks across turns
 - `context-manipulation.json` - 30 payloads for conversation history attacks
@@ -23,18 +36,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Updated corpus sizes: quick=100, standard=400, thorough=800, exhaustive=1500
 
 ##### Usage Telemetry (P0)
+
 - `src/telemetry/usage.ts` - Event tracking with privacy controls
 - `src/telemetry/registry.ts` - Persistent scan registry for analytics
 - Opt-in telemetry for repo URL, org name, user email
 - Analytics methods for dashboard and case study candidates
 
 ##### Badge Service (P0)
+
 - `src/badge-service/index.ts` - HTTP handlers for badge serving
 - Badge verification endpoint with Sigstore bundle support
 - `generateBadgeEmbedCode()` for markdown/HTML embedding
 - CertificationStorage interface with memory implementation
 
 ##### Frontier Model Interface (P1)
+
 - `src/frontier/types.ts` - Interfaces for Mythos/GPT-5.5-Cyber integration
 - `src/frontier/orchestrator.ts` - Multi-model orchestration with consensus
 - `src/frontier/providers/stub.ts` - Test provider placeholder
@@ -42,6 +58,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - ExploitChain and ConsensusResult types
 
 ##### Data Flow Analysis (P1)
+
 - `src/analysis/data-flow.ts` - Source→sink tracking for JS/TS/Python
 - Pattern-based detection of user input sources (req.body, event.body, etc.)
 - Dangerous sink detection (SQL, command exec, eval, file write)
@@ -49,6 +66,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - LLM context formatting for focused analysis
 
 ##### Agent Chain Analysis (P2)
+
 - `src/scanners/agent/agent-chain-analysis.ts` - Multi-hop attack paths
 - Trust boundary modeling between agents and MCP servers
 - AgentGraph construction from MCP server configs
@@ -56,6 +74,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Mermaid diagram generation for visualization
 
 ### Changed
+
 - Extended PayloadCategory type with 7 new categories
 - Updated FuzzerOptions corpus type to include "exhaustive"
 - Increased test count from 2,332 to 2,484 across 104 test files
@@ -65,6 +84,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 #### Agent Batch Submit Tool
+
 - New `agent_batch_submit` tool for submitting findings from subagent JSON output
 - Solves MCP permission issues when certification agents run as subagents
 - Accepts array of findings and optional summary in one call
@@ -73,6 +93,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 #### CI/CD Improvements
+
 - Lazy Stripe initialization to allow builds without `STRIPE_SECRET_KEY`
 - Fixed TypeScript test timeout for CI environments
 - Synced package-lock.json for CI compatibility
@@ -82,6 +103,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 #### Plan Enforcement
+
 - New plan-limits system for free/pro/enterprise tiers
 - Certification monthly limits enforced at API level
 - Agent count limits based on subscription plan
@@ -90,19 +112,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 #### Plan Limits
 
-| Limit | Free | Pro | Enterprise |
-|-------|------|-----|------------|
-| Certifications/month | 3 | 50 | Unlimited |
-| Projects | 2 | 20 | Unlimited |
-| Agents | 3 | 7 | All |
-| Frameworks | SOC2 | SOC2, HIPAA, NIST | All |
-| Red team | ❌ | ❌ | ✓ |
+| Limit                | Free | Pro               | Enterprise |
+| -------------------- | ---- | ----------------- | ---------- |
+| Certifications/month | 3    | 50                | Unlimited  |
+| Projects             | 2    | 20                | Unlimited  |
+| Agents               | 3    | 7                 | All        |
+| Frameworks           | SOC2 | SOC2, HIPAA, NIST | All        |
+| Red team             | ❌   | ❌                | ✓          |
 
 ## [2.6.0] - 2026-04-26
 
 ### Added
 
 #### Test Coverage
+
 - 147 new tests across 5 test files
 - `agent-integrity.test.ts` - Consensus analysis and outlier detection
 - `agent-privacy.test.ts` - PII detection with Luhn validation
@@ -111,12 +134,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `flags.test.ts` - Feature flags and config loading
 
 #### Feature Flags System
+
 - New `.vaspera/config.yaml` configuration format
 - Per-agent weights and model selection
 - Per-scanner timeouts and custom rules
 - Feature toggles for multiModel, costTracking, autofix, etc.
 
 #### Plugin System
+
 - Scanner plugin architecture with manifest schema
 - Local plugins from `.vaspera/plugins/`
 - npm plugins from `vaspera-scanner-*` packages
@@ -127,6 +152,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 #### Mythos-Class Security Scanners
+
 - New `binary-analysis` scanner for native module security
   - Detects Node.js native addons, shared libraries, Rust FFI, Go CGO
   - Checks RELRO, NX, PIE, CANARY protections via checksec
@@ -144,6 +170,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Java: check-then-act and synchronized patterns
 
 #### Semantic AI Agents
+
 - New `zero-day-hunter` agent for novel vulnerability discovery
   - AI-powered semantic code analysis beyond pattern matching
   - Discovers logic flaws, auth bypasses, cryptographic weaknesses
@@ -160,17 +187,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Severity escalation calculation (medium + medium = critical)
 
 #### New MCP Tools
+
 - `certification_scan_binary` - Scan compiled code and native modules
 - `certification_analyze_chains` - Analyze findings for exploitable chains
 - `certification_semantic_analysis` - Run AI-powered semantic analysis
 
 #### Compliance Enhancements
+
 - Added MITRE ATT&CK technique mapping for AI/ML systems
 - New CWE mappings for memory safety vulnerabilities
 - New CWE mappings for race condition vulnerabilities
 - OWASP LLM Top 10 integration
 
 ### Changed
+
 - Updated scanner count from 9 to 13+ scanners
 - Updated agent count from 4 to 7+ agents
 - Updated frontend marketing pages with Mythos-class capabilities
@@ -181,6 +211,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 #### Cost Tracking
+
 - New `cost_track` tool to start tracking costs for a certification
 - New `cost_estimate` tool to estimate costs before running
 - New `cost_status` tool to get current cost status
@@ -191,6 +222,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Budget limits with automatic warnings and abort capability
 
 #### Multi-Model Consensus
+
 - New `multimodel_record` tool to record findings from model runs
 - New `multimodel_consensus` tool to calculate inter-model agreement
 - New `multimodel_disagreements` tool to identify model disagreements
@@ -203,6 +235,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Disagreement detection by type (existence, severity, location, description)
 
 #### Compliance Mapping
+
 - New `compliance_report` tool for single-framework reports
 - New `compliance_multi_report` tool for multi-framework reports
 - New `compliance_controls` tool to list framework controls
@@ -212,6 +245,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Finding-to-control mapping by category
 
 #### SBOM & Provenance
+
 - New `sbom_generate` tool for CycloneDX SBOM generation
 - New `sbom_provenance` tool for SLSA provenance attestation
 - New `sbom_sign` tool for Sigstore signing
@@ -220,6 +254,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Build attestation with SLSA Level 2 support
 
 #### Documentation
+
 - New `docs/` folder with feature documentation
 - Cost tracking guide (`docs/cost-tracking.md`)
 - Multi-model consensus guide (`docs/multi-model.md`)
@@ -228,11 +263,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Example workflows (`docs/examples/`)
 
 ### Changed
+
 - Updated MCP tool count from 36 to 52
 - Updated package description to highlight enterprise features
 - README now includes v2.0.0 features section
 
 ### Fixed
+
 - Finding type now uses `description` consistently (removed legacy `title`)
 - Multi-model consensus correctly handles partial model agreement
 - Cost calculation uses accurate per-model pricing
@@ -242,58 +279,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 #### Deterministic Scanners
+
 - Semgrep integration for OWASP Top 10
 - gitleaks integration for secrets detection
 - npm audit integration for CVE detection
 - TypeScript analysis for type safety
 
 #### GitHub Action
+
 - `action.yml` for CI/CD integration
 - Diff-mode scanning for PRs
 - PR comment formatting
 - SARIF upload to GitHub Code Scanning
 
 #### Evaluation Harness
+
 - Test fixtures for scanner accuracy
 - Precision, recall, F1 metrics
 - Stability testing across runs
 - Target thresholds for publication
 
 #### Custom Rules
+
 - `rules_load` for custom rule loading
 - `rules_templates` for built-in templates
 - `rules_generate_config` for config generation
 - `rules_check_file` for file checking
 
 ### Changed
+
 - Scanner findings now have confidence: 100
 - LLM agents reference scanner findings by ID
 
 ## [1.0.2] - 2023-12-15
 
 ### Added
+
 - Cross-verification system between agents
 - Consensus scoring with certification levels
 - SARIF export for GitHub integration
 
 ### Fixed
+
 - Evidence validation for LLM findings
 - Finding deduplication logic
 
 ## [1.0.1] - 2023-12-01
 
 ### Added
+
 - File hash-based caching
 - Agent finding submission tools
 - Basic certification workflow
 
 ### Fixed
+
 - Project discovery on macOS
 - Command installation paths
 
 ## [1.0.0] - 2023-11-15
 
 ### Added
+
 - Initial release
 - 6 certification agents (security, reliability, typesafety, performance, quality, redteam)
 - Hardening command installation

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AAwPpE;;GAEG;AACH,iBAAS,YAAY,CAAC,IAAI,EAAE,MAAM;;;;;EAIjC;AAED;;GAEG;AACH,iBAAS,YAAY,CAAC,CAAC,SAAS,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC;;;;;;EAK/D;AAED;;GAEG;AACH,iBAAS,aAAa,CAAC,OAAO,EAAE,MAAM;;;;;EAIrC;AA+CD,QAAA,MAAM,MAAM,WAGV,CAAC;AAghKH,OAAO,EAAE,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AAiQpE;;GAEG;AACH,iBAAS,YAAY,CAAC,IAAI,EAAE,MAAM;;;;;EAIjC;AAED;;GAEG;AACH,iBAAS,YAAY,CAAC,CAAC,SAAS,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC;;;;;;EAK/D;AAED;;GAEG;AACH,iBAAS,aAAa,CAAC,OAAO,EAAE,MAAM;;;;;EAIrC;AA+CD,QAAA,MAAM,MAAM,WAGV,CAAC;AAkoKH,OAAO,EAAE,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC"}

package/dist/index.js

@@ -46,6 +46,9 @@ import { getTracker, formatCost, formatTokens, estimateCost, getSupportedModels,
 import { getRunner, DEFAULT_MODELS, formatProvider, } from "./multimodel/index.js";
 // Path validation utilities
 import { validateProjectPath, PathValidationError } from "./util/paths.js";
+// Telemetry and scan registry
+import { trackCertificationStarted, trackCertificationCompleted, trackScannerRun, } from "./telemetry/usage.js";
+import { getRegistry } from "./telemetry/registry.js";
 // ---------------------------------------------------------------------------
 // Config
 // ---------------------------------------------------------------------------
@@ -755,6 +758,12 @@ server.registerTool("certification_scan", {
         project: basename(project_path),
     });
     scanLogger.info("scanners.starting", { scanners, auto_detect });
+    const startTime = Date.now();
+    // Track scan start via telemetry
+    const scannersToRun = auto_detect
+        ? ["auto-detect"]
+        : Object.entries(scanners || {}).filter(([, v]) => v).map(([k]) => k);
+    await trackCertificationStarted(project_path, scannersToRun, [], auto_detect ? "auto" : "manual");
     // Use auto-detection or manual scanner selection
     let result;
     let detectedLanguages;
@@ -768,6 +777,24 @@ server.registerTool("certification_scan", {
     else {
         result = await runAllScanners(project_path, scanners);
     }
+    // Track scanner runs in telemetry
+    for (const scanner of Object.keys(result.byScanner)) {
+        await trackScannerRun(project_path, scanner, result.totalDuration / Object.keys(result.byScanner).length, // Approximate per-scanner duration
+        result.byScanner[scanner] || 0, !result.failedScanners.includes(scanner));
+    }
+    // Record scan in registry for analytics
+    const registry = getRegistry();
+    await registry.recordScan({
+        certificationId: certification_id,
+        projectPath: project_path,
+        scanDate: new Date().toISOString(),
+        duration: Date.now() - startTime,
+        findingsSummary: result.bySeverity,
+        totalFindings: result.totalFindings,
+        scannersRun: Object.keys(result.byScanner),
+        frameworksAssessed: [],
+        success: result.allSucceeded,
+    });
     // If certification_id provided and submit_findings is true, submit to certification
     if (certification_id && submit_findings && result.totalFindings > 0) {
         const certFindings = scannerFindingsToCertificationFindings(result);
@@ -1579,6 +1606,7 @@ server.registerTool("certification_finalize", {
     },
 }, async ({ project_path, certification_id }) => {
     const certLogger = createChildLogger({ certId: certification_id, project: basename(project_path) });
+    const startTime = Date.now();
     const certification = await getCertification(project_path, certification_id);
     if (!certification) {
         certLogger.warn("certification.not_found");
@@ -1604,6 +1632,36 @@ server.registerTool("certification_finalize", {
     }
     // Generate artifacts
     const artifacts = await writeCertificationArtifacts(project_path, finalCert);
+    // Track certification completion via telemetry
+    const severityCounts = { critical: 0, high: 0, medium: 0, low: 0, info: 0 };
+    let totalFindings = 0;
+    for (const agentType of Object.keys(certification.agents || {})) {
+        const agent = certification.agents[agentType];
+        if (agent?.findings) {
+            for (const finding of agent.findings) {
+                severityCounts[finding.severity]++;
+                totalFindings++;
+            }
+        }
+    }
+    await trackCertificationCompleted(project_path, certification_id, finalCert.consensus?.certification_level || "BLOCKED", finalCert.consensus?.overall_score || 0, Date.now() - new Date(certification.metadata.started_at).getTime(), severityCounts, totalFindings, [] // frameworks
+    );
+    // Record in registry
+    const registry = getRegistry();
+    await registry.recordScan({
+        certificationId: certification_id,
+        projectPath: project_path,
+        scanDate: new Date().toISOString(),
+        level: finalCert.consensus?.certification_level || "BLOCKED",
+        score: finalCert.consensus?.overall_score || 0,
+        duration: Date.now() - startTime,
+        findingsSummary: severityCounts,
+        totalFindings,
+        scannersRun: Object.keys(certification.agents || {}),
+        frameworksAssessed: [],
+        success: true,
+        tags: ["certification-finalized"],
+    });
     certLogger.info("certification.finalized", {
         level: finalCert.consensus?.certification_level,
         score: finalCert.consensus?.overall_score,
@@ -3625,7 +3683,11 @@ Maps findings to AI compliance frameworks (OWASP LLM, NIST AI RMF, EU AI Act).`,
     if (!authorized) {
         return errorResponse("Agent scanning requires explicit authorization. Set authorized=true to confirm you have permission to scan this target.");
     }
+    const startTime = Date.now();
     try {
+        // Track scan start via telemetry
+        const enabledScanners = scanners || AGENT_SCANNER_TYPES;
+        await trackCertificationStarted(target, enabledScanners, frameworks || [], "agent-cert");
         // Build scan target
         const scanTarget = {};
         if (target.startsWith("http://") || target.startsWith("https://")) {
@@ -3641,7 +3703,6 @@ Maps findings to AI compliance frameworks (OWASP LLM, NIST AI RMF, EU AI Act).`,
             scanTarget.npmPackage = target;
         }
         // Build scanner options
-        const enabledScanners = scanners || AGENT_SCANNER_TYPES;
         const scannerFlags = {
             manifestAudit: enabledScanners.includes("manifest-audit"),
             toolDrift: enabledScanners.includes("tool-description-drift"),
@@ -3680,6 +3741,28 @@ Maps findings to AI compliance frameworks (OWASP LLM, NIST AI RMF, EU AI Act).`,
         }
         // Generate summary
         const summary = generateAgentScannerSummary(result);
+        // Record scan in registry for analytics
+        const registry = getRegistry();
+        await registry.recordScan({
+            certificationId: certification_id,
+            projectPath: target,
+            scanDate: new Date().toISOString(),
+            level: result.certificationReadiness === "ready" ? "CERTIFIED"
+                : result.certificationReadiness === "needs-review" ? "REVIEW_REQUIRED"
+                    : "BLOCKED",
+            score: 100 - result.riskScore,
+            duration: Date.now() - startTime,
+            findingsSummary: result.bySeverity,
+            totalFindings: result.totalFindings,
+            scannersRun: result.scanners.map((s) => s.scanner),
+            frameworksAssessed: frameworks || [],
+            success: result.allSucceeded,
+            tags: ["agent-cert", "mcp-security"],
+        });
+        // Track individual scanner runs
+        for (const scanner of result.scanners) {
+            await trackScannerRun(target, scanner.scanner, scanner.duration || 0, scanner.findings.length, scanner.success);
+        }
         return jsonResponse({
             success: result.allSucceeded,
             target,