adaptive-memory-multi-model-router 2.13.17 → 2.13.20

package/.github/PULL_REQUEST_TEMPLATE.md

@@ -1,44 +1,71 @@
-## 📝 Description
+## Description
 
-Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
+Please include a summary of the change and which issue it fixes. Explain the motivation and context — what problem does this solve and why is this the right approach?
 
 Fixes # (issue)
 
-## 🔄 Type of Change
+## Type of Change
 
-Please delete options that are not relevant.
+Check all that apply:
 
-- [ ] 🐛 Bug fix (non-breaking change which fixes an issue)
-- [ ] ✨ New feature (non-breaking change which adds functionality)
-- [ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
-- [ ] 📚 Documentation update
-- [ ] 🎨 Code style/formatting
-- [ ] ⚡ Performance improvement
-- [ ] 🔒 Security improvement
+- [ ] 🐛 **Bug fix** (non-breaking change that fixes an issue)
+- [ ] ✨ **New feature** (non-breaking change that adds functionality)
+- [ ] 💥 **Breaking change** (fix or feature that would break existing functionality)
+- [ ] 📚 **Documentation** (README, API docs, inline comments, or examples)
+- [ ] 🎨 **Style** (formatting, code style — no logic change)
+- [ ] ⚡ **Performance** (improves speed, memory, or latency without behavior change)
+- [ ] 🔒 **Security** (fixes a vulnerability or improves security posture)
+- [ ] ♻️ **Refactor** (code change that neither fixes a bug nor adds a feature)
+- [ ] 🧪 **Tests** (adds or updates tests, not production code)
 
-## 🧪 Testing
+## Testing
 
-Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.
+Describe the tests you ran to verify your changes:
 
 - [ ] All existing tests pass (`npm test`)
 - [ ] Added new tests for the changes
-- [ ] Tested manually with example code
+- [ ] Tested manually (describe the commands or SDK calls used)
+- [ ] Tested with real provider API calls (if applicable)
 
-## 📋 Checklist
+```bash
+# Provide commands to reproduce your testing
+node test/some-test.js
+npx a3m-router route "Test query"
+```
 
-- [ ] My code follows the style guidelines of this project
+## Checklist
+
+- [ ] My code follows the project's coding style and conventions
 - [ ] I have performed a self-review of my own code
-- [ ] I have commented my code, particularly in hard-to-understand areas
-- [ ] I have made corresponding changes to the documentation
-- [ ] My changes generate no new warnings
-- [ ] I have added tests that prove my fix is effective or that my feature works
-- [ ] New and existing unit tests pass locally with my changes
-- [ ] Any dependent changes have been merged and published in downstream modules
+- [ ] I have commented complex logic, especially in hard-to-understand areas
+- [ ] I have updated the documentation (README, API docs, JSDoc comments) if needed
+- [ ] My changes produce no new warnings (no `console.log` left behind, no TypeScript errors)
+- [ ] I have added tests that prove my fix is effective or my feature works
+- [ ] New and existing unit tests pass locally
+- [ ] Any dependent changes are merged and published in downstream modules
+- [ ] I have checked that my changes are backward-compatible (or documented breaking changes)
+
+## Screenshots (if applicable)
+
+If the change affects the terminal UI, dashboard, or any visual output, include screenshots:
+
+| Before | After |
+|--------|-------|
+| _screenshot_ | _screenshot_ |
+
+## Additional Context
+
+- Related issues or PRs
+- Performance impact (latency, memory, bundle size)
+- Configuration changes required
+- Migration guide (if breaking change)
 
-## 📸 Screenshots (if applicable)
+## Changelog Entry
 
-Add screenshots to help explain your changes.
+Suggest a one-line changelog entry for this change:
 
-## 💬 Additional Comments
+```
+- {type}: {short description} ({issue/PR link})
+```
 
-Add any other context or screenshots about the pull request here.
+Example: `- feat: add language-based routing override (#42)`

package/.github/dependabot.yml

@@ -0,0 +1,9 @@
+version: 2
+updates:
+  - package-ecosystem: "npm"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    open-pull-requests-limit: 10
+    labels:
+      - "dependencies"

package/.github/workflows/codeql.yml

@@ -0,0 +1,38 @@
+name: "CodeQL"
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  schedule:
+    - cron: '0 0 * * 0'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    strategy:
+      fail-fast: false
+      matrix:
+        language: ['javascript-typescript']
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v3
+      with:
+        languages: ${{ matrix.language }}
+
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v3
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v3

package/.github/workflows/npm-publish.yml

@@ -0,0 +1,20 @@
+name: Publish to npm
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+          registry-url: 'https://registry.npmjs.org'
+      - run: npm ci
+      - run: npm run build
+      - run: npm publish
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

package/.github/workflows/stale.yml

@@ -0,0 +1,56 @@
+# Stale Issue and PR Management
+# Marks issues and PRs as stale after 60 days of inactivity,
+# then closes them 7 days later if no further activity.
+#
+# https://github.com/actions/stale
+
+name: "Stale Issue & PR"
+on:
+  schedule:
+    - cron: "0 6 * * 1" # Every Monday at 6:00 UTC
+
+permissions:
+  issues: write
+  pull-requests: write
+
+jobs:
+  stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v9
+        with:
+          # -- General Configuration --
+          days-before-stale: 60
+          days-before-close: 7
+          operations-per-run: 100
+
+          # -- Messaging --
+          stale-issue-message: >
+            This issue has been automatically marked as stale because it has been
+            inactive for 60 days. If this is still relevant, please leave a comment
+            to keep it open. Otherwise, it will be closed in 7 days.
+          close-issue-message: >
+            This issue has been automatically closed due to inactivity.
+            Feel free to reopen if the issue persists or is still relevant.
+          stale-pr-message: >
+            This pull request has been automatically marked as stale because it has
+            been inactive for 60 days. Please update or comment to keep it active.
+            It will be closed in 7 days.
+          close-pr-message: >
+            This pull request has been automatically closed due to inactivity.
+            Feel free to reopen with updates when you're ready to continue.
+
+          # -- Labels --
+          stale-issue-label: stale
+          stale-pr-label: stale
+          exempt-issue-labels: "pinned,security,blocked,awaiting-review"
+          exempt-pr-labels: "pinned,security,blocked,awaiting-review"
+          exempt-all-milestones: true
+
+          # -- Exempt from stale --
+          # Never mark these as stale
+          exempt-issue-labels: "pinned,security,blocked,awaiting-review,help-wanted,good-first-issue"
+          exempt-pr-labels: "pinned,security,blocked,awaiting-review,WIP"
+
+          # -- Delete stale branches on close --
+          delete-branch: false

package/ARCHITECTURE.md

@@ -0,0 +1,346 @@
+# Architecture
+
+## A3M Router — Adaptive Memory Multi-Model Router
+
+A multi-provider LLM routing and orchestration engine. Routes prompts across 47+ providers, executes them in parallel with ensemble voting, and adapts model selection based on learned quality profiles, cost constraints, and task complexity.
+
+## High-Level Overview
+
+The system has three layers:
+
+```
+User / API / CLI / TUI
+        |
+  [Proxy Server / LangChain Adapter]
+        |
+   [Routing Engine]  ←── [Memory System]  ←── [Semantic Cache]
+        |
+   [Provider Layer]  ←── [Retry Handler]  ←── [Guardrails]
+        |
+   [47+ LLM APIs]
+```
+
+- **TypeScript Core** — routing, provider config, cost tracking, observability, cache, guardrails, proxy server, TUI
+- **Python Layer** — Universal Model Router (learned routing), HALO orchestration (hierarchical planning), MCTS workflow search
+- **Integrations** — LangChain adapter, MCP server, OpenAI-compatible proxy, CLI/TUI
+
+## Directory Structure
+
+```
+src/
+  index.ts                 # Main entry point — exports all public APIs, createA3MRouter()
+  sdk.ts                   # A3MRouter SDK class — route(), routeBatch(), recommend(), serve(), analyze()
+  routing/
+    providerRetry.ts       # Per-provider retry with exponential backoff + jitter, context window validation
+    providerHealth.ts      # Provider health monitoring
+    universal_router.py    # UniversalModelRouter — learned routing with online adaptation (Python)
+  providers/
+    providerConfig.ts      # 47+ provider definitions, config loading, health checks, runtime registration
+    registry.py            # Python provider registry with health monitoring
+    base.py                # Python base provider classes
+    anthropic.py           # Anthropic provider implementation (Python)
+    cerebras.py            # Cerebras provider implementation (Python)
+  memory/
+    memoryTree.ts          # MemoryTree — hierarchical chunk storage with search
+    autoFetch.ts           # Automatic memory fetching
+    obsidianVault.ts       # Obsidian vault integration
+    agentic_memory.py      # Agentic memory (Python)
+    semantic_memory.py     # Semantic memory (Python)
+    simple_memory.py       # Simple memory (Python)
+    working_memory.py      # Working memory (Python)
+  cost/
+    costTracker.ts         # Per-request cost tracking
+    budgetEnforcer.ts      # Budget limits, spend records, alerts
+  analytics/
+    costAnalytics.ts       # Advanced cost analytics, savings reports, projections
+  cache/
+    semanticCache.ts       # Embedding-based semantic cache with cosine similarity
+    research/              # Cache research files
+  security/
+    guardrails.ts          # Prompt injection, PII detection, content filtering, output validation
+  observability/
+    index.ts               # Observable exports
+    types.ts               # Span, Metric, RouteTrace types
+    tracer.ts              # Distributed tracing
+    metrics.ts             # Metrics collector
+    middleware.ts          # Express-style observability middleware
+  server/
+    proxyServer.ts         # OpenAI-compatible HTTP proxy — POST /v1/chat/completions, GET /v1/models
+    modelMapper.ts         # Model name resolution
+    dashboard.ts           # Server dashboard
+  integrations/
+    langchainAdapter.ts    # Drop-in ChatOpenAI replacement for LangChain
+    oauth.ts               # OAuth integration
+  cli/
+    setupWizard.ts         # Interactive setup wizard
+  tui/
+    index.ts               # TUI launch wrapper
+    dashboard.ts           # Blessed-based terminal dashboard
+  orchestration/           # (Python) HALO hierarchical orchestration
+    halo_orchestrator.py   # HALO orchestrator — 3-tier planning
+    task_planner.py        # Task decomposition into subtasks
+    role_assigner.py       # Agent role assignment
+    execution_engine.py    # Parallel execution with verification
+    mcts_workflow.py       # MCTS-based workflow search
+  workflows/               # (Python) Workflow executors
+    router.py              # Workflow router
+    orchestrator.py        # Workflow orchestrator
+    chaining_executor.py   # Sequential chain execution
+    parallelization_executor.py  # Parallel task execution
+    difficulty_integration.py    # Difficulty-aware routing
+  agents/
+    skill_enhanced_agent.py  # Skill-enhanced agent (Python)
+  state/
+    simple_checkpoint.py     # State checkpointing (Python)
+  types/
+    langchain.d.ts        # LangChain type declarations
+  utils/                   # (referenced from index.ts exports)
+    tokenUtils.ts          # Token counting and estimation
+
+python/
+  a3m/                    # Python SDK for A3M Router
+  tmlpd.py                # TMLPD Python client
+  examples.py             # Usage examples
+  integrations.py         # Python integration helpers
+
+mcp-server/               # MCP (Model Context Protocol) server for AI agent integration
+integrations/             # Additional integration entry points
+eval/                     # Evaluation framework and benchmarks
+test/ tests/              # Test suites (TypeScript + Python)
+docs/                     # GitHub Pages documentation site
+demo/                     # Demo scripts and recordings
+```
+
+## Key Components
+
+### 1. Ensemble Voting (P0)
+
+The unique differentiator. Routes the same query to multiple providers in parallel, then merges responses using confidence-weighted voting. No other LLM router does this — everyone does sequential fallback (try A, then B, then C).
+
+The ensemble flow:
+1. Query enters the routing engine
+2. Classifier extracts features (complexity, domain, length, code/math presence)
+3. Top-N candidate models selected by tier, cost, and quality profile
+4. Query dispatched to all N providers in parallel
+5. Responses collected and merged with confidence weighting
+6. Best merged result returned with fallback alternatives
+
+### 2. Query Classification
+
+The routing engine (`sdk.ts` → `extractQueryFeatures`) classifies queries on 10+ signals:
+
+| Signal | Description |
+|--------|-------------|
+| complexity | 0.0–1.0, based on keyword density and reasoning indicators |
+| has_code | Code block or programming keyword presence |
+| has_math | Mathematical expression detection |
+| is_multilingual | Non-English character ratio |
+| is_translation | Translation verb detection |
+| is_creative | Creative writing indicators |
+| requires_reasoning | Step-by-step reasoning triggers |
+| domain | Detected domain (legal, medical, security, finance, devops, data) |
+
+Classification routes to the `free` / `cheap` / `mid` / `premium` cost tier, targeting 99.5% accuracy within +/-1 tier (validated by independent benchmark).
+
+### 3. Memory System
+
+The `MemoryTree` (`memory/memoryTree.ts`) canonicalizes data into ≤3k-token chunks, scores each by relevance, and builds hierarchical summary trees. Supports:
+- **Search**: keyword matching with score ranking
+- **Context retrieval**: top-scored chunks for routing enrichment
+- **Obsidian export**: markdown serialization
+- **Stats**: tree depth, chunk count, memory utilization
+
+Python memory variants (`agentic_memory.py`, `semantic_memory.py`, `working_memory.py`) provide agent-specific memory stores for the orchestration layer.
+
+### 4. Provider Routing
+
+The provider system (`providers/providerConfig.ts`) defines 47+ providers across five tiers:
+
+| Tier | Providers | Purpose |
+|------|-----------|---------|
+| free | Ollama, LM Studio, vLLM, Google (free tier), NVIDIA NIM | Local / zero-cost |
+| cheap | Groq, Cerebras, DeepInfra, Together, Fireworks, Novita, SambaNova, Anyscale, Replicate | Inference-optimized |
+| mid | DeepSeek, Mistral, Perplexity, Cohere, AI21, Qwen (DashScope), StepFun | Good quality/price |
+| premium | OpenAI, Anthropic, xAI (Grok) | Frontier models |
+| enterprise | Azure OpenAI, AWS Bedrock, Google Vertex | Cloud-managed |
+
+Each provider has:
+- `baseUrl`, `apiKeyEnv` (env var name), `models` list
+- `costPerK` (input/output), `tier`, `format` (openai/anthropic/google/cohere/aws-bedrock/google-vertex)
+- `type` (api/cli/local), `priority` (selection order), `maxTokens`
+
+Configuration sources (in priority order):
+1. Environment variables (`*_API_KEY`)
+2. `~/.config/a3m-router/providers.json`
+3. Runtime registration via `registerProvider()`
+
+### 5. Security (Guardrails Engine)
+
+The `GuardrailEngine` (`security/guardrails.ts`) provides configurable input/output checks:
+- **Prompt injection**: score-based detection (0–100)
+- **PII detection and redaction**: emails, phones, SSNs, credit cards, IPs
+- **Content filtering**: configurable blocklist, regex patterns
+- **Language detection**: for intelligent routing decisions
+- **Output validation**: quality checks, hallucination detection
+- **Custom guardrails**: user-defined check functions
+
+### 6. Observability
+
+Three subsystems:
+- **Tracer**: distributed tracing with span creation, completion, and route trace construction
+- **MetricsCollector**: runtime metrics — request counts, latencies, error rates, cache hit rates
+- **Middlewares**: Express-style `observabilityMiddleware`, `observabilityPlugin`, `budgetAlertMiddleware`
+
+### 7. Semantic Cache
+
+Embedding-based cache (`cache/semanticCache.ts`) stores query-response pairs. On lookup, computes cosine similarity against stored embeddings. Supports configurable threshold (default 0.92), TTL, LRU eviction (1000 entries), and multiple embedders (nomic via Ollama, OpenAI, or local).
+
+### 8. Cost Tracking
+
+Tiered cost management:
+- **CostTracker**: per-request recording with provider, model, tokens, latency
+- **CostAnalytics**: savings reports, monthly projections, provider breakdowns, CSV/JSON export
+- **BudgetEnforcer**: hard budget caps with pre-request checks and alerts
+
+### 9. Proxy Server
+
+OpenAI-compatible HTTP proxy (`server/proxyServer.ts`) using only Node.js built-in `http` module. Endpoints:
+- `POST /v1/chat/completions` — OpenAI-compatible chat
+- `POST /v1/completions` — Text completions
+- `GET /v1/models` — Available models
+- `GET /health` — Provider health status
+
+Any OpenAI SDK can point to this proxy to get A3M routing automatically.
+
+### 10. HALO Orchestration (Python)
+
+Hierarchical Autonomous Logic-Oriented Orchestration based on arXiv:2505.13516. Three tiers:
+
+1. **TaskPlanner**: decomposes complex tasks into subtasks with dependency resolution
+2. **RoleAssigner**: assigns specialized agents (roles) to each subtask
+3. **ExecutionEngine**: executes subtasks in parallel with verification and adaptive refinement
+
+Optionally uses **MCTS** (Monte Carlo Tree Search) to explore different execution strategies and learn optimal workflows per task type.
+
+### 11. MCP Server
+
+Model Context Protocol server for AI agent integration. Allows AI agents (Claude, etc.) to use the A3M Router as a tool for parallel multi-LLM execution.
+
+### 12. LangChain Integration
+
+`A3MChatModel` (`integrations/langchainAdapter.ts`) is a drop-in replacement for `ChatOpenAI`. Routes all LLM calls through A3M for cost optimization and intelligent provider selection. Supports streaming, tool calling, and batch processing.
+
+## Data Flow
+
+```
+1. User sends query (via SDK, proxy, CLI, TUI, or LangChain)
+2. GuardrailsEngine checks input (injection, PII, content, length)
+3. SemanticCache looks up embedding match (skip if cache hit)
+4. RoutingEngine classifies query (complexity, domain, features)
+5. Router selects optimal provider(s) using:
+   - Learned quality profiles (UniversalModelRouter)
+   - Cost constraints (BudgetEnforcer)
+   - Retry configuration (ProviderRetryHandler)
+6. (Optional) Multiple providers called in parallel for ensemble voting
+7. ProviderRetryHandler executes with exponential backoff + jitter
+8. GuardrailsEngine validates output
+9. Response returned + recorded in:
+   - CostTracker (per-request cost)
+   - CostAnalytics (aggregate stats)
+   - Observability (tracing + metrics)
+   - SemanticCache (store for future hits)
+   - MemoryTree (context enrichment)
+```
+
+## Design Decisions and Trade-offs
+
+| Decision | Rationale | Trade-off |
+|----------|-----------|-----------|
+| **TypeScript primary** | npm ecosystem reach, serverless compatibility, Vercel/Netlify/Cloudflare Workers | Python users need separate SDK |
+| **Node.js built-in http** for proxy | Zero dependencies, 19.5 KB total bundle | Less feature-rich than Express |
+| **Embedding-based cache** | Semantic similarity beats exact-match for LLM queries | Requires Ollama or OpenAI embedder |
+| **Per-provider retry config** | Chinese providers need longer timeouts + more retries (network latency, rate limits) | More config surface |
+| **In-memory storage** | Zero infra, instant setup, 19.5 KB | No persistence across restarts (memory tree serializable to markdown) |
+| **Online learning (Python router)** | Adapts to unseen models and changing quality | Requires feedback loop, cold start with heuristics |
+| **MCTS for workflow search** | Finds optimal strategies for complex tasks | 3-10x slower than greedy for simple tasks |
+| **47+ baked-in providers** | Zero-config multi-provider out of box | Maintenance burden as APIs change |
+
+## Extension Points
+
+### Adding a New Provider
+
+```typescript
+import { registerProvider, ProviderDefinition } from 'adaptive-memory-multi-model-router';
+
+registerProvider('my-provider', {
+  name: 'My Provider',
+  baseUrl: 'https://api.myprovider.com/v1/chat/completions',
+  apiKeyEnv: 'MY_PROVIDER_API_KEY',
+  models: ['model-name'],
+  costPerK: { input: 1.0, output: 2.0 },
+  tier: 'mid',       // free | cheap | mid | premium | enterprise
+  format: 'openai',  // openai | anthropic | google | cohere | aws-bedrock | google-vertex
+  type: 'api',       // api | cli | local
+  priority: 15,
+  maxTokens: 8192,
+});
+```
+
+Or via config file at `~/.config/a3m-router/providers.json`:
+```json
+{
+  "providers": {
+    "my-provider": {
+      "name": "My Provider",
+      "baseUrl": "https://api.myprovider.com/v1/chat/completions",
+      "apiKeyEnv": "MY_PROVIDER_API_KEY",
+      "models": ["model-name"],
+      "tier": "mid"
+    }
+  }
+}
+```
+
+### Adding a Custom Retry Strategy
+
+```typescript
+import { createRetryHandler } from 'adaptive-memory-multi-model-router';
+
+const handler = createRetryHandler({
+  'my-slow-provider': {
+    timeout: 60000,
+    retry: { maxRetries: 5, initialDelayMs: 5000 },
+  },
+});
+```
+
+### Adding Custom Guardrails
+
+```typescript
+import { GuardrailEngine } from 'adaptive-memory-multi-model-router';
+
+const guardrails = new GuardrailEngine({
+  userGuardrails: [
+    (content) => ({
+      passed: !content.includes('blocked-term'),
+      blocked: content.includes('blocked-term'),
+      reason: content.includes('blocked-term') ? 'Blocked term detected' : undefined,
+    }),
+  ],
+});
+```
+
+### Adding Ensemble Voting Strategies
+
+The ensemble system is extensible by adding new voting strategies to the parallel execution pipeline. Current strategy: confidence-weighted average across multiple provider responses.
+
+### Custom Routing Strategies
+
+The `UniversalModelRouter` (Python) learns routing profiles from execution data. To implement a custom strategy:
+1. Subclass or wrap `routeQuery` in TypeScript
+2. Or extend `UniversalModelRouter._calculate_combined_score` in Python
+3. Register custom feature extractors via `extractQueryFeatures`
+
+### MCP Server Extensions
+
+The MCP server at `mcp-server/` exposes routing as tools. Add tools by extending the MCP tool definitions.

package/AUDIT_REPORT.md

@@ -0,0 +1,28 @@
+# Dependency Audit Report
+
+**Date:** 2026-05-28
+**Project:** adaptive-memory-multi-model-router (v2.13.18)
+
+## Summary
+
+| Metric | Value |
+|--------|-------|
+| Total dependencies | 5 (3 prod + 2 dev) |
+| Vulnerabilities | 0 (none found) |
+| Outdated packages | 1 updated |
+
+## Vulnerabilities
+
+**0 vulnerabilities found.** The dependency tree is clean with no reported security issues across all direct and transitive dependencies.
+
+## Updated Packages
+
+| Package | From | To | Type | Reason |
+|---------|------|----|------|--------|
+| `@types/node` | 25.8.0 | 25.9.1 | devDependency | Updated via `npm update` within `^25.8.0` semver range |
+
+## Notes
+
+- `@langchain/core` is listed as `MISSING` in `npm outdated` output — this is expected. It is an **optional peer dependency** (`"optional": true` in `peerDependenciesMeta`) and is not required for core functionality.
+- All other dependencies (`blessed@0.1.81`, `nanoid@5.1.11`, `typescript@6.0.3`) are up-to-date within their semver ranges.
+- No breaking changes were introduced — `npm update` only applied compatible version bumps within declared semver ranges.