agentic-flow 1.9.0 → 1.9.1

package/docs/releases/archive/GITHUB-ISSUE-BUILD-FIX.md

@@ -1,756 +0,0 @@
-# GitHub Issue: Complete Fix for Native Build Dependency Errors
-
-## Issue Title
-[FIXED] Native build errors with better-sqlite3 and sharp in npx installations (GitHub Codespaces, Docker, CI/CD)
-
-## Labels
-- `bug`
-- `dependencies`
-- `build`
-- `docker`
-- `codespaces`
-- `fixed`
-
----
-
-## Problem Description
-
-### Summary
-Users installing `agentic-flow` and `agentdb` via `npx` in environments without native build tools (Python, Make, G++, SQLite development libraries) were experiencing installation failures due to native dependencies attempting to compile.
-
-### Error Examples
-
-**Error 1: better-sqlite3 compilation failure**
-```bash
-$ npx agentic-flow@1.8.1
-npm ERR! code ENOENT
-npm ERR! syscall spawn sh
-npm ERR! path /home/codespace/.npm/_npx/.../node_modules/better-sqlite3
-npm ERR! errno -2
-npm ERR! enoent spawn sh ENOENT
-```
-
-**Error 2: sharp (image processing) compilation failure**
-```bash
-$ npx agentic-flow@1.8.1
-npm ERR! code ENOENT
-npm ERR! syscall spawn sh
-npm ERR! path /home/codespace/.npm/_npx/.../node_modules/sharp
-npm ERR! errno -2
-npm ERR! enoent spawn sh ENOENT
-```
-
-### Affected Environments
-- ✅ GitHub Codespaces (no build tools by default)
-- ✅ Docker containers with slim/alpine images
-- ✅ CI/CD pipelines without build tool installations
-- ✅ Cloud development environments
-- ✅ Windows systems without Visual Studio Build Tools
-- ✅ macOS systems without Xcode Command Line Tools
-
-### Root Causes
-
-The errors were caused by THREE layers of native dependencies:
-
-#### 1. Direct Dependencies (Fixed in agentdb v1.3.16 → v1.4.3)
-```json
-// agentdb package.json (BEFORE)
-{
-  "optionalDependencies": {
-    "better-sqlite3": "^11.8.1",      // ❌ Requires Python, Make, G++, SQLite
-    "@xenova/transformers": "^2.17.2"  // ❌ Depends on sharp (native)
-  }
-}
-```
-
-**Issue**: Even as `optionalDependencies`, npm still ATTEMPTS to install and build them with `npx`, causing failures.
-
-#### 2. Transitive Dependencies (Fixed in agentic-flow v1.8.3)
-```
-agentic-flow@1.8.1
-├── claude-flow@2.0.0
-│   └── ruv-swarm@1.0.20
-│       └── better-sqlite3@11.10.0  ← ❌ HIDDEN DEPENDENCY
-└── agentic-payments@0.1.3
-    └── agentic-flow@1.6.4  ← ❌ CIRCULAR DEP with old version
-        └── better-sqlite3 (via claude-flow)
-```
-
-**Issue**: Even after removing better-sqlite3 from agentdb, it was still being pulled in through dependency chains.
-
-#### 3. Source Code Imports (Fixed in both packages)
-```typescript
-// src/memory/SharedMemoryPool.ts
-import Database from 'better-sqlite3';  // ❌ Direct import
-
-// src/reasoningbank/db/queries.ts
-import BetterSqlite3, { Database } from 'better-sqlite3';  // ❌ Direct import
-```
-
-**Issue**: TypeScript compilation failed when better-sqlite3 wasn't available.
-
----
-
-## Solution Overview
-
-We implemented a **three-phase fix** to completely eliminate native build dependencies while maintaining full functionality.
-
-### Phase 1: AgentDB Migration to sql.js (v1.4.3)
-
-Migrated from better-sqlite3 (native) to sql.js (WebAssembly).
-
-**Why sql.js?**
-- ✅ Pure JavaScript/WebAssembly - no compilation needed
-- ✅ Works in ANY environment (browsers, Node.js, Deno, Bun)
-- ✅ Same SQLite compatibility
-- ✅ No system dependencies required
-- ⚠️ Slightly slower than native (acceptable for agent memory use cases)
-
-### Phase 2: Dependency Tree Cleanup (agentic-flow v1.8.3)
-
-Removed packages that brought in native dependencies through transitive relationships.
-
-### Phase 3: Source Code Migration
-
-Updated all source files to use sql.js wrapper instead of better-sqlite3 direct imports.
-
----
-
-## Detailed Fix Implementation
-
-### Step 1: AgentDB v1.4.3 - Remove better-sqlite3
-
-#### 1.1: Update package.json
-```bash
-cd packages/agentdb
-
-# Remove better-sqlite3 from dependencies
-npm uninstall better-sqlite3 @types/better-sqlite3
-
-# Remove @xenova/transformers (depends on sharp)
-npm uninstall @xenova/transformers
-```
-
-**Result: packages/agentdb/package.json**
-```json
-{
-  "name": "agentdb",
-  "version": "1.4.3",
-  "dependencies": {
-    "@modelcontextprotocol/sdk": "^1.20.1",
-    "chalk": "^5.3.0",
-    "commander": "^12.1.0",
-    "sql.js": "^1.13.0",      // ✅ WASM SQLite - no build tools
-    "zod": "^3.25.76"
-  },
-  "optionalDependencies": {},  // ✅ EMPTY - no optional native deps
-  "keywords": [
-    "agentdb",
-    "vector-database",
-    "sqlite",
-    "sql.js",                  // ✅ Added
-    "wasm"                     // ✅ Added
-    // "better-sqlite3"        // ❌ REMOVED
-  ]
-}
-```
-
-#### 1.2: Update db-fallback.ts (sql.js only)
-```bash
-# Edit: packages/agentdb/src/db-fallback.ts
-```
-
-**Before:**
-```typescript
-export async function getDatabaseImplementation() {
-  try {
-    // Try better-sqlite3 first
-    const BetterSqlite3 = await import('better-sqlite3');
-    return BetterSqlite3.default;
-  } catch {
-    // Fallback to sql.js
-    const SQL = await (await import('sql.js')).default();
-    return createSqlJsWrapper(SQL);
-  }
-}
-```
-
-**After:**
-```typescript
-export async function getDatabaseImplementation(): Promise<any> {
-  // ONLY sql.js - no better-sqlite3 attempt
-  try {
-    console.log('✅ Using sql.js (WASM SQLite, no build tools required)');
-
-    const mod = await import('sql.js');
-    const SQL = await mod.default();
-
-    return createSqlJsWrapper(SQL);
-  } catch (error) {
-    throw new Error(
-      'Failed to initialize SQLite. Please ensure sql.js is installed:\n' +
-      'npm install sql.js'
-    );
-  }
-}
-```
-
-#### 1.3: Update all TypeScript imports
-```bash
-# Fix all files importing better-sqlite3
-cd packages/agentdb/src
-
-# Update CLI
-# src/cli/agentdb-cli.ts
-# BEFORE: import Database from 'better-sqlite3';
-# AFTER:  import { createDatabase } from '../db-fallback.js';
-
-# Update controllers (replace type imports)
-for file in controllers/*.ts; do
-  sed -i "s/import { Database } from 'better-sqlite3'/\/\/ Database type from db-fallback\ntype Database = any/" "$file"
-done
-
-# Update MCP server
-# src/mcp/agentdb-mcp-server.ts
-# BEFORE: import Database from 'better-sqlite3';
-# AFTER:  type Database = any;
-```
-
-#### 1.4: Update EmbeddingService for optional transformers
-```bash
-# Edit: packages/agentdb/src/controllers/EmbeddingService.ts
-```
-
-**Before:**
-```typescript
-import { pipeline } from '@xenova/transformers';
-
-async initialize() {
-  this.pipeline = await pipeline('feature-extraction', this.config.model);
-}
-```
-
-**After:**
-```typescript
-async initialize() {
-  if (this.config.provider === 'transformers') {
-    try {
-      // Dynamic import for optional dependency
-      const { pipeline } = await import('@xenova/transformers' as any);
-      this.pipeline = await pipeline('feature-extraction', this.config.model);
-    } catch (error) {
-      console.warn('Transformers.js not available, falling back to mock embeddings');
-      this.pipeline = null;
-    }
-  }
-}
-```
-
-#### 1.5: Build and publish
-```bash
-cd packages/agentdb
-
-# Clean build
-rm -rf node_modules package-lock.json dist
-npm install
-npm run build
-
-# Verify no better-sqlite3 in node_modules
-npm ls better-sqlite3
-# Should show: (empty)
-
-# Publish
-npm publish --access public
-```
-
-**Published: agentdb@1.4.3**
-- ✅ Zero native dependencies
-- ✅ Works in all environments
-- ✅ All features functional (reflexion memory, causal recall, skill library)
-
----
-
-### Step 2: agentic-flow v1.8.3 - Remove Transitive Dependencies
-
-#### 2.1: Identify problematic dependencies
-```bash
-cd agentic-flow
-
-# Check dependency tree for better-sqlite3
-npm ls better-sqlite3
-
-# Output shows:
-# agentic-flow@1.8.1
-# ├── claude-flow@2.0.0
-# │   └── ruv-swarm@1.0.20
-# │       └── better-sqlite3@11.10.0  ← FOUND IT!
-# └── agentic-payments@0.1.3
-#     └── agentic-flow@1.6.4  ← Circular dependency
-```
-
-#### 2.2: Check if dependencies are actually used
-```bash
-# Search source code for imports
-grep -r "from 'claude-flow'" src/ --include="*.ts"
-# Result: (empty) - NOT USED IN SOURCE
-
-grep -r "from 'agentic-payments'" src/ --include="*.ts"
-# Result: Only in CLI help text - NOT CRITICAL
-```
-
-#### 2.3: Remove unused dependencies
-```bash
-# Edit package.json
-nano package.json
-```
-
-**Before:**
-```json
-{
-  "version": "1.8.2",
-  "dependencies": {
-    "agentdb": "^1.4.2",
-    "agentic-payments": "^0.1.3",  // ❌ Brings old agentic-flow
-    "claude-flow": "^2.0.0",       // ❌ Brings ruv-swarm → better-sqlite3
-    "axios": "^1.12.2"
-  }
-}
-```
-
-**After:**
-```json
-{
-  "version": "1.8.3",
-  "dependencies": {
-    "agentdb": "^1.4.3",          // ✅ Updated to sql.js version
-    "axios": "^1.12.2"
-    // ✅ claude-flow REMOVED
-    // ✅ agentic-payments REMOVED
-  }
-}
-```
-
-#### 2.4: Update source code imports
-```bash
-# Fix remaining better-sqlite3 imports in agentic-flow source
-
-# src/memory/SharedMemoryPool.ts
-# BEFORE: import Database from 'better-sqlite3';
-# AFTER:  type Database = any;
-
-# src/reasoningbank/db/queries.ts
-# BEFORE: import BetterSqlite3, { Database } from 'better-sqlite3';
-# AFTER:  type Database = any;
-#         const BetterSqlite3: any = null;
-```
-
-#### 2.5: Handle optional ONNX dependencies (TypeScript stubs)
-```bash
-# Create stub type definitions for optional deps
-mkdir -p src/types
-
-# src/types/onnxruntime-node.d.ts
-cat > src/types/onnxruntime-node.d.ts << 'EOF'
-// Stub types for optional dependency onnxruntime-node
-declare module 'onnxruntime-node' {
-  export namespace Tensor {
-    const type: any;
-  }
-  export class InferenceSession {
-    static create(path: string, options?: any): Promise<any>;
-    run(feeds: any, options?: any): Promise<any>;
-    [key: string]: any;
-  }
-  export const env: any;
-}
-EOF
-
-# src/types/huggingface-inference.d.ts
-cat > src/types/huggingface-inference.d.ts << 'EOF'
-// Stub types for optional dependency @huggingface/inference
-declare module '@huggingface/inference' {
-  export class HfInference {
-    constructor(token?: string);
-    [key: string]: any;
-  }
-}
-EOF
-```
-
-#### 2.6: Update build script for lenient compilation
-```bash
-# Edit package.json scripts section
-```
-
-**Before:**
-```json
-{
-  "scripts": {
-    "build": "npm run build:wasm && tsc -p config/tsconfig.json && cp -r src/reasoningbank/prompts dist/reasoningbank/"
-  }
-}
-```
-
-**After:**
-```json
-{
-  "scripts": {
-    "build": "npm run build:wasm && tsc -p config/tsconfig.json --skipLibCheck || true && cp -r src/reasoningbank/prompts dist/reasoningbank/"
-  }
-}
-```
-
-**Why `--skipLibCheck || true`?**
-- Skips type checking for node_modules (optional deps may be missing)
-- `|| true` ensures build continues even with TypeScript warnings
-- Still validates your own source code
-
-#### 2.7: Clean install and verify
-```bash
-# Remove all dependencies and reinstall
-rm -rf node_modules package-lock.json
-npm install
-
-# Verify NO native dependencies
-npm ls better-sqlite3 sharp @xenova/transformers
-# All should show: (empty)
-
-# Check total dependency tree
-npm ls | grep -i "sqlite\|sharp\|onnx\|xenova" || echo "✅ Clean!"
-```
-
-#### 2.8: Build and publish
-```bash
-npm run build
-npm publish --access public
-```
-
-**Published: agentic-flow@1.8.3**
-- ✅ Zero native dependencies in dependency tree
-- ✅ Uses agentdb@1.4.3 (sql.js)
-- ✅ Works in all environments
-
----
-
-## Verification Testing
-
-### Test 1: Clean npx Installation (GitHub Codespaces)
-```bash
-# Clear npm cache
-npm cache clean --force
-
-# Test agentdb
-npx agentdb@latest db stats
-
-# Expected output:
-# ✅ Using sql.js (WASM SQLite, no build tools required)
-# 📊 Database Statistics
-# ════════════════════════════════════════════════════════════════
-# causal_edges: 0 records
-# episodes: 0 records
-# ════════════════════════════════════════════════════════════════
-
-# Test agentic-flow
-npx agentic-flow@latest --version
-
-# Expected output:
-# agentic-flow v1.8.3
-
-# ✅ NO BUILD ERRORS!
-```
-
-### Test 2: Docker Installation (Slim Image)
-```dockerfile
-# Dockerfile.test
-FROM node:22-slim
-
-# NO build tools installed
-# RUN apt-get update && apt-get install -y python3 make g++ sqlite3 libsqlite3-dev
-
-WORKDIR /app
-
-# Test installation
-RUN npx agentdb@latest db stats
-RUN npx agentic-flow@latest --version
-
-CMD ["echo", "✅ All packages installed successfully!"]
-```
-
-```bash
-# Build and run
-docker build -f Dockerfile.test -t test-zero-build .
-docker run --rm test-zero-build
-
-# Expected output:
-# ✅ Using sql.js (WASM SQLite, no build tools required)
-# agentic-flow v1.8.3
-# ✅ All packages installed successfully!
-```
-
-### Test 3: Verify Dependency Tree
-```bash
-# Create test directory
-mkdir -p /tmp/test-deps && cd /tmp/test-deps
-
-# Install latest versions
-npm init -y
-npm install agentdb@latest agentic-flow@latest
-
-# Check for native dependencies
-npm ls | grep -E "better-sqlite3|sharp|@xenova|onnxruntime" || echo "✅ No native deps!"
-
-# Verify package.json
-npm view agentdb@latest dependencies
-npm view agentic-flow@latest dependencies
-
-# Expected: NO better-sqlite3, sharp, or @xenova/transformers
-```
-
-### Test 4: Functionality Verification
-```bash
-# Test AgentDB features
-npx agentdb@latest db init
-
-# Store reflexion memory
-npx agentdb@latest reflexion store \
-  --session "test-session" \
-  --task "Test task" \
-  --input "Test input" \
-  --output "Test output" \
-  --reward 0.9
-
-# Retrieve memory
-npx agentdb@latest reflexion search \
-  --session "test-session" \
-  --k 5
-
-# Expected: Successfully stores and retrieves data using sql.js
-
-# Test skill library
-npx agentdb@latest skills create \
-  --name "test-skill" \
-  --description "Test skill" \
-  --code "console.log('test')"
-
-npx agentdb@latest skills search \
-  --query "test" \
-  --k 5
-
-# Expected: Successfully manages skills using sql.js
-```
-
----
-
-## Performance Impact
-
-### Benchmark Results
-
-**better-sqlite3 (native) vs sql.js (WASM):**
-
-| Operation | better-sqlite3 | sql.js | Ratio |
-|-----------|---------------|--------|-------|
-| Insert (1k rows) | 12ms | 45ms | 3.75x slower |
-| Query (simple) | 0.8ms | 2.1ms | 2.6x slower |
-| Query (complex join) | 15ms | 58ms | 3.9x slower |
-| Vector search | 25ms | 92ms | 3.7x slower |
-
-**Analysis:**
-- ✅ sql.js is 2.6-3.9x slower than native
-- ✅ **Still acceptable** for agent memory use cases (ms vs seconds)
-- ✅ **Worth the tradeoff** for zero-build universal compatibility
-- ✅ Can upgrade to better-sqlite3 manually if needed for production
-
-### Memory Usage
-- sql.js: ~8MB WASM bundle
-- better-sqlite3: ~500KB native binary
-- **Tradeoff**: Larger bundle but no compilation needed
-
----
-
-## Migration Guide for Users
-
-### For Users on Old Versions
-
-#### Option 1: Update to Latest (Recommended)
-```bash
-# Clear cache
-npm cache clean --force
-
-# Update globally
-npm install -g agentdb@latest agentic-flow@latest
-
-# Or use npx (always latest)
-npx agentdb@latest db stats
-npx agentic-flow@latest --version
-```
-
-#### Option 2: Manual Installation (if needed)
-```bash
-# Install in project
-npm install agentdb@1.4.3 agentic-flow@1.8.3
-
-# Verify no native deps
-npm ls better-sqlite3 sharp
-# Should show: (empty)
-```
-
-### For Users Who Want Native Performance
-
-If you have build tools and want better-sqlite3 performance:
-
-```bash
-# Clone repository
-git clone https://github.com/ruvnet/agentic-flow.git
-cd agentic-flow/packages/agentdb
-
-# Install WITH optional native deps
-npm install better-sqlite3
-
-# The code will auto-detect and use better-sqlite3 if available
-npm run build
-npm test
-
-# Verify it's using native
-# Should see: "✅ Using better-sqlite3 (native, high performance)"
-```
-
----
-
-## Breaking Changes
-
-### agentdb v1.4.3
-- ✅ **NO BREAKING CHANGES** - API identical
-- ✅ Automatic fallback maintains compatibility
-- ⚠️ Slightly slower performance (acceptable for most use cases)
-
-### agentic-flow v1.8.3
-- ⚠️ **REMOVED**: `claude-flow` dependency
-  - **Impact**: claude-flow MCP tools not available by default
-  - **Workaround**: Install claude-flow separately if needed
-- ⚠️ **REMOVED**: `agentic-payments` dependency
-  - **Impact**: Payment tools not available by default
-  - **Workaround**: Install agentic-payments separately if needed
-- ✅ **All core features** still functional
-
----
-
-## Files Changed
-
-### agentdb v1.4.3
-```
-packages/agentdb/
-├── package.json                          # Removed better-sqlite3, @xenova/transformers
-├── src/
-│   ├── db-fallback.ts                    # sql.js only implementation
-│   ├── cli/agentdb-cli.ts                # Use createDatabase()
-│   ├── controllers/
-│   │   ├── EmbeddingService.ts           # Optional transformers import
-│   │   ├── CausalMemoryGraph.ts          # Type Database = any
-│   │   ├── CausalRecall.ts               # Type Database = any
-│   │   ├── ExplainableRecall.ts          # Type Database = any
-│   │   ├── LearningSystem.ts             # Type Database = any
-│   │   ├── NightlyLearner.ts             # Type Database = any
-│   │   ├── ReasoningBank.ts              # Type Database = any
-│   │   ├── ReflexionMemory.ts            # Type Database = any
-│   │   └── SkillLibrary.ts               # Type Database = any
-│   ├── mcp/agentdb-mcp-server.ts         # Type Database = any
-│   └── optimizations/
-│       ├── BatchOperations.ts            # Type Database = any
-│       └── QueryOptimizer.ts             # Type Database = any
-```
-
-### agentic-flow v1.8.3
-```
-agentic-flow/
-├── package.json                          # Removed claude-flow, agentic-payments
-├── src/
-│   ├── memory/SharedMemoryPool.ts        # Type Database = any
-│   ├── reasoningbank/db/queries.ts       # Type Database = any
-│   └── types/
-│       ├── onnxruntime-node.d.ts         # NEW: Stub types
-│       └── huggingface-inference.d.ts    # NEW: Stub types
-```
-
----
-
-## Testing Checklist
-
-- [x] npx installation in GitHub Codespaces (no build tools)
-- [x] npx installation in Docker slim image (no build tools)
-- [x] npm install in fresh project
-- [x] Dependency tree verification (no better-sqlite3, no sharp)
-- [x] AgentDB CLI functionality (db stats, reflexion, skills, causal)
-- [x] agentic-flow CLI functionality (version, help)
-- [x] MCP server initialization
-- [x] Database operations (create, insert, query, update, delete)
-- [x] Vector search functionality
-- [x] Memory persistence across sessions
-- [x] Embedding service fallback behavior
-- [x] TypeScript compilation (with --skipLibCheck)
-- [x] Package publishing to npm registry
-
----
-
-## Related Issues
-
-- #XXX: better-sqlite3 installation failures in Docker
-- #XXX: GitHub Codespaces build errors
-- #XXX: Windows installation requires Visual Studio
-- #XXX: macOS installation requires Xcode tools
-
----
-
-## References
-
-### Documentation
-- [sql.js Documentation](https://sql.js.org/)
-- [better-sqlite3 Build Requirements](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/troubleshooting.md)
-- [npm optionalDependencies](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#optionaldependencies)
-
-### Commits
-- agentdb v1.4.3: `[commit-hash]` - Remove better-sqlite3, use sql.js exclusively
-- agentic-flow v1.8.3: `[commit-hash]` - Remove transitive native dependencies
-
-### Published Packages
-- https://www.npmjs.com/package/agentdb/v/1.4.3
-- https://www.npmjs.com/package/agentic-flow/v/1.8.3
-
----
-
-## Resolution Status
-
-**Status**: ✅ **RESOLVED**
-
-**Fixed in**:
-- agentdb@1.4.3 (published)
-- agentic-flow@1.8.3 (published)
-
-**Verification**:
-```bash
-npx agentdb@latest db stats      # ✅ Works
-npx agentic-flow@latest --version # ✅ Works
-```
-
-**No build tools required!** 🎉
-
----
-
-## Questions or Issues?
-
-If you still experience build errors after updating to these versions:
-
-1. Clear npm cache: `npm cache clean --force`
-2. Verify versions: `npm view agentdb version` (should be 1.4.3+)
-3. Check dependency tree: `npm ls better-sqlite3` (should be empty)
-4. Report detailed error with:
-   - Operating system and version
-   - Node.js version: `node --version`
-   - npm version: `npm --version`
-   - Full error output
-   - Output of: `npm ls | grep -E "sqlite|sharp"`
-
----
-
-**Maintainer**: @ruvnet
-**Last Updated**: 2025-10-25
-**Fixed Versions**: agentdb@1.4.3, agentic-flow@1.8.3