agentic-flow 1.0.0

package/docs/integrations/fastmcp-poc-integration.md

@@ -0,0 +1,198 @@
+# FastMCP POC Integration Guide
+
+## Overview
+
+The FastMCP POC provides a minimal stdio-based MCP server with 2 basic tools (`memory_store`, `memory_retrieve`) to validate the fastmcp framework integration with agentic-flow.
+
+## Architecture
+
+```
+agentic-flow/
+├── src/mcp/fastmcp/
+│   ├── servers/
+│   │   └── poc-stdio.ts      # POC server (stdio transport)
+│   ├── tools/
+│   │   └── memory/
+│   │       ├── store.ts      # Memory store tool
+│   │       └── retrieve.ts   # Memory retrieve tool
+│   ├── types/
+│   │   └── index.ts          # TypeScript definitions
+│   └── config/
+│       └── mcp-config.json   # MCP client configuration
+```
+
+## Installation
+
+Already installed as part of agentic-flow:
+```bash
+npm install  # fastmcp & zod included
+npm run build
+```
+
+## Usage
+
+### Option 1: Direct CLI Test
+```bash
+# Run test script
+npm run test:fastmcp
+
+# Or manually:
+npm run mcp:fastmcp-poc
+```
+
+### Option 2: Claude Code Integration
+Add to Claude Code's MCP settings (`~/.config/claude/mcp.json`):
+```json
+{
+  "mcpServers": {
+    "fastmcp-poc": {
+      "command": "node",
+      "args": ["/workspaces/flow-cloud/docker/claude-agent-sdk/dist/mcp/fastmcp/servers/poc-stdio.js"],
+      "env": {}
+    }
+  }
+}
+```
+
+### Option 3: NPX Integration
+```bash
+# From package root
+npx agentic-flow mcp:fastmcp-poc
+```
+
+## Available Tools
+
+### 1. memory_store
+Store a value in persistent memory via claude-flow.
+
+**Parameters:**
+- `key` (string, required): Memory key
+- `value` (string, required): Value to store
+- `namespace` (string, optional): Memory namespace (default: "default")
+- `ttl` (number, optional): Time-to-live in seconds
+
+**Example:**
+```json
+{
+  "name": "memory_store",
+  "arguments": {
+    "key": "test-key",
+    "value": "test-value",
+    "namespace": "poc-test",
+    "ttl": 3600
+  }
+}
+```
+
+**Returns:**
+```json
+{
+  "success": true,
+  "key": "test-key",
+  "namespace": "poc-test",
+  "size": 10,
+  "ttl": 3600,
+  "timestamp": "2025-10-03T20:41:00.000Z",
+  "message": "Memory stored successfully"
+}
+```
+
+### 2. memory_retrieve
+Retrieve a value from persistent memory.
+
+**Parameters:**
+- `key` (string, required): Memory key
+- `namespace` (string, optional): Memory namespace (default: "default")
+
+**Example:**
+```json
+{
+  "name": "memory_retrieve",
+  "arguments": {
+    "key": "test-key",
+    "namespace": "poc-test"
+  }
+}
+```
+
+**Returns:**
+```json
+{
+  "success": true,
+  "key": "test-key",
+  "namespace": "poc-test",
+  "value": "test-value",
+  "timestamp": "2025-10-03T20:42:00.000Z"
+}
+```
+
+## Testing
+
+### Automated Test
+```bash
+npm run test:fastmcp
+```
+
+### Manual Test with MCP Protocol
+```bash
+# Test memory_store
+echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"memory_store","arguments":{"key":"test","value":"hello","namespace":"poc"}}}' | npm run mcp:fastmcp-poc
+
+# Test memory_retrieve
+echo '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"memory_retrieve","arguments":{"key":"test","namespace":"poc"}}}' | npm run mcp:fastmcp-poc
+```
+
+## Integration Status
+
+✅ **Phase 0 Complete** - POC with stdio transport
+- [x] Created POC server with 2 tools
+- [x] Implemented stdio transport
+- [x] Tools use claude-flow backend via execSync
+- [x] TypeScript compilation successful
+- [x] Test scripts created
+- [x] MCP configuration provided
+
+🔄 **Phase 1 Pending** - In-process migration (claude-flow-sdk)
+- [ ] Migrate 6 claude-flow-sdk tools to fastmcp
+- [ ] Remove execSync, use direct imports
+- [ ] Validate all tools work in stdio
+
+## Known Limitations
+
+1. **Backend Dependency**: Currently calls `npx claude-flow@alpha` via execSync
+   - Phase 1 will replace with direct imports for in-process execution
+
+2. **Progress Reporting**: Removed temporarily (fastmcp API investigation needed)
+   - Context.onProgress not available in current fastmcp API
+
+3. **Error Handling**: Basic throw-based errors
+   - Will be enhanced with structured error types in Phase 1
+
+## Next Steps (Phase 1)
+
+1. Migrate claude-flow-sdk tools (6 tools):
+   - memory_store, memory_retrieve, memory_search
+   - swarm_init, agent_spawn, task_orchestrate
+
+2. Replace execSync with direct imports:
+   ```typescript
+   // Before (Phase 0):
+   const result = execSync(`npx claude-flow@alpha memory store ...`);
+
+   // After (Phase 1):
+   import { MemoryManager } from '../../memory/manager.js';
+   const memory = new MemoryManager();
+   const result = await memory.store(key, value, namespace, ttl);
+   ```
+
+3. Add progress reporting when fastmcp API clarified
+
+4. Implement comprehensive error types
+
+## Resources
+
+- FastMCP Implementation Plan: `docs/mcp/fastmcp-implementation-plan.md`
+- POC Server: `src/mcp/fastmcp/servers/poc-stdio.ts`
+- Tool Definitions: `src/mcp/fastmcp/tools/memory/`
+- Test Script: `scripts/test-fastmcp-poc.sh`
+- MCP Config: `src/mcp/fastmcp/config/mcp-config.json`

package/docs/router/ONNX_PHI4_RESEARCH.md

@@ -0,0 +1,220 @@
+# ONNX Runtime Integration Research - Phi-4 Implementation
+
+**Date**: 2025-10-03
+**Status**: Research Complete - Implementation Ready
+
+## Executive Summary
+
+Research findings for integrating Microsoft's Phi-4-mini-instruct-onnx model with agentic-flow using onnxruntime-node for CPU inference.
+
+## Key Findings
+
+### 1. Library Comparison
+
+| Library | Use Case | Performance | Node.js Support | Status |
+|---------|----------|-------------|-----------------|--------|
+| **onnxruntime-node** | Server-side inference | **Fastest** | ✅ Native | **Recommended** |
+| onnxruntime-web | Browser/frontend | Good | ✅ WebAssembly | Not suitable for CLI |
+| @xenova/transformers | Simplified API | Moderate | ✅ Yes | Limited model support |
+| onnxruntime-genai | Generative AI | Excellent | ❌ Python only | Not available for Node.js |
+
+**Conclusion**: Use **onnxruntime-node** v1.22.0 - it's the official Microsoft library with best performance for server-side Node.js applications.
+
+### 2. Phi-4-mini-instruct-onnx Model Details
+
+**HuggingFace**: https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx
+
+#### Model Specifications
+- **Context Length**: 128K tokens
+- **License**: MIT
+- **Quantization**: INT4 (RTN - Round To Nearest)
+- **Variants**:
+  - `cpu-int4-rtn-block-32-acc-level-4` - CPU optimized
+  - `gpu-int4-rtn-block-32` - CUDA optimized
+
+#### Performance Characteristics
+- **Speedup**: 12.4x faster than PyTorch on CPU
+- **Memory**: Reduced via INT4 quantization
+- **Platform**: Cross-platform (Windows, Linux, macOS)
+
+### 3. Key Challenges Identified
+
+#### Challenge 1: onnxruntime-genai Not Available for Node.js
+- **Issue**: The `onnxruntime-genai` library (used in Python examples) has no npm package
+- **Impact**: Cannot use simplified GenAI API in Node.js
+- **Solution**: Use onnxruntime-node directly with manual tokenization
+
+#### Challenge 2: Transformers.js Incompatibility
+- **Issue**: @xenova/transformers doesn't support Phi-4 models (only GPT-2, Llama, etc.)
+- **Error**: "Unsupported model type: phi3"
+- **Solution**: Bypass transformers.js, use onnxruntime-node + custom tokenizer
+
+#### Challenge 3: Manual Tokenization Required
+- **Issue**: Need to implement Phi-4 chat template and tokenization
+- **Required**:
+  - Tokenizer model (tokenizer.json)
+  - Chat template formatting
+  - Pre/post processing
+- **Solution**: Use HuggingFace tokenizers library or implement manually
+
+### 4. Recommended Architecture
+
+```
+┌─────────────────────────────────────────────┐
+│         ONNXProvider (Updated)              │
+├─────────────────────────────────────────────┤
+│                                             │
+│  ┌───────────────────────────────────┐     │
+│  │   onnxruntime-node v1.22.0       │     │
+│  │   (InferenceSession)              │     │
+│  └───────────────────────────────────┘     │
+│              ↓                              │
+│  ┌───────────────────────────────────┐     │
+│  │   Phi-4 ONNX Model                │     │
+│  │   cpu-int4-rtn-block-32           │     │
+│  └───────────────────────────────────┘     │
+│              ↓                              │
+│  ┌───────────────────────────────────┐     │
+│  │   Custom Tokenizer                │     │
+│  │   (Phi-4 chat template)           │     │
+│  └───────────────────────────────────┘     │
+│                                             │
+└─────────────────────────────────────────────┘
+```
+
+### 5. Implementation Plan
+
+#### Phase 1: Download Phi-4 Model ✅
+```bash
+huggingface-cli download microsoft/Phi-4-mini-instruct-onnx \
+  --include cpu-int4-rtn-block-32-acc-level-4/* \
+  --local-dir ./models/phi-4
+```
+
+#### Phase 2: Install Dependencies
+```bash
+npm install onnxruntime-node@^1.22.0
+npm install @huggingface/tokenizers  # For tokenization
+```
+
+#### Phase 3: Implement ONNXProvider
+- Use onnxruntime-node InferenceSession API
+- Load Phi-4 ONNX model from disk
+- Implement Phi-4 chat template
+- Handle tokenization/detokenization
+- Support CPU execution provider (upgradeable to CUDA)
+
+#### Phase 4: Chat Template Format
+```typescript
+// Phi-4 uses the following chat format:
+// <|system|>
+// {system_message}<|end|>
+// <|user|>
+// {user_message}<|end|>
+// <|assistant|>
+// {assistant_response}<|end|>
+```
+
+### 6. API Comparison
+
+#### Python (onnxruntime-genai) - NOT AVAILABLE FOR NODE
+```python
+import onnxruntime_genai as og
+model = og.Model("cpu-int4-rtn-block-32-acc-level-4")
+tokenizer = og.Tokenizer(model)
+params = og.GeneratorParams(model)
+generator = og.Generator(model, params)
+```
+
+#### Node.js (onnxruntime-node) - RECOMMENDED
+```typescript
+import * as ort from 'onnxruntime-node';
+
+// Load model
+const session = await ort.InferenceSession.create(
+  './models/phi-4/model.onnx',
+  { executionProviders: ['cpu'] }
+);
+
+// Manual tokenization
+const inputIds = tokenize(prompt);
+const feeds = { input_ids: new ort.Tensor('int64', inputIds, [1, inputIds.length]) };
+
+// Run inference
+const results = await session.run(feeds);
+const outputIds = results.output.data;
+const text = detokenize(outputIds);
+```
+
+### 7. Performance Targets
+
+| Metric | Target | Expected |
+|--------|--------|----------|
+| **First Token Latency** | <2000ms | ~1500ms |
+| **Tokens/Second** | >15 | 15-25 |
+| **Memory Usage** | <4GB | ~2-3GB |
+| **Cost** | $0 | FREE |
+
+### 8. Execution Providers
+
+| Provider | Platform | Support | Acceleration |
+|----------|----------|---------|--------------|
+| CPU | All | ✅ Default | AVX2, AVX512 |
+| CUDA | Linux + NVIDIA | ✅ Available | 10-100x |
+| DirectML | Windows | ✅ Available | 5-20x |
+| CoreML | macOS | ⚠️ Experimental | 5-10x |
+
+### 9. Model Download Strategy
+
+**Option 1: Manual Download (Recommended)**
+- Use huggingface-cli to pre-download model
+- Store in `./models/phi-4/` directory
+- Faster initialization, no runtime downloads
+
+**Option 2: Automatic Download**
+- Use @huggingface/hub library
+- Download on first run
+- Slower first initialization
+
+**Recommendation**: Pre-download for Docker deployments, auto-download for development.
+
+### 10. Docker Considerations
+
+```dockerfile
+# In Dockerfile
+RUN pip install huggingface-hub
+RUN huggingface-cli download microsoft/Phi-4-mini-instruct-onnx \
+    --include cpu-int4-rtn-block-32-acc-level-4/* \
+    --local-dir /app/models/phi-4
+
+# Or mount as volume
+volumes:
+  - ./models:/app/models
+```
+
+## Next Steps
+
+1. ✅ Research complete - onnxruntime-node confirmed as best option
+2. 🔄 Download Phi-4 model files
+3. ⏳ Implement ONNXProvider with onnxruntime-node
+4. ⏳ Create tokenizer integration
+5. ⏳ Test in Docker CPU environment
+6. ⏳ Benchmark performance
+7. ⏳ Add GPU support (CUDA/DirectML)
+
+## Resources
+
+- ONNX Runtime Node.js: https://onnxruntime.ai/docs/api/js/
+- Phi-4 Model: https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx
+- ONNX Runtime GenAI: https://github.com/microsoft/onnxruntime-genai (Python reference)
+- HuggingFace Tokenizers: https://www.npmjs.com/package/@huggingface/tokenizers
+
+## Conclusion
+
+**onnxruntime-node is the correct choice** for implementing Phi-4 inference in agentic-flow:
+- Official Microsoft library
+- Best performance for Node.js
+- CPU and GPU support
+- Production-ready
+
+**Note**: We'll need to implement manual tokenization since onnxruntime-genai (with built-in tokenization) is Python-only.