adaptive-memory-multi-model-router 2.14.46 → 2.14.48

package/test-council/council.md

@@ -1,183 +0,0 @@
-# A3M Router Test Council
-
-## Overview
-
-The A3M Router Test Council is a multi-agent testing approach inspired by the agent council pattern used in production LLM systems. Instead of a single monolithic test suite, we use **3 specialist agents**, each evaluating the codebase from a distinct perspective.
-
-## Why Agent Council?
-
-Traditional testing treats all tests equally. A council approach recognizes that:
-
-1. **Different aspects of code quality require different testing strategies**
-2. **Specialized agents can find issues a general test would miss**
-3. **Council decisions are more robust than single-agent decisions**
-
-## The 3 Specialist Agents
-
-### 1. Structure Agent (`1-structure-tests.ts`)
-
-**Focus:** Code architecture, type safety, and export validation
-
-**Specialization:**
-- Verifies all exports are present and correctly typed
-- Validates provider configuration structure
-- Checks model profile completeness
-- Ensures class instantiation works
-- Validates enum and constant values
-
-**Key Questions:**
-- Are all public APIs exported correctly?
-- Do model profiles have required fields?
-- Are provider configs valid?
-- Do class constructors work?
-
-**Example Findings:**
-- Missing required fields in provider config
-- Inconsistent type definitions
-- Invalid enum values
-- Missing method exports
-
-### 2. Edge-Case Agent (`2-edge-case-tests.ts`)
-
-**Focus:** Boundary conditions, error handling, and invalid inputs
-
-**Specialization:**
-- Tests empty and null inputs
-- Validates extremely long inputs
-- Checks unknown/invalid models
-- Tests missing API key handling
-- Verifies concurrent request safety
-- Tests special characters and unicode
-- Validates error recovery
-
-**Key Questions:**
-- What happens with empty strings?
-- How does the system handle null inputs?
-- What about 10,000 word queries?
-- Does concurrent access cause race conditions?
-- Are special characters handled safely?
-
-**Example Findings:**
-- Null pointer exceptions on empty inputs
-- Buffer overflow on very long queries
-- Race conditions in shared state
-- Missing error boundaries
-
-### 3. Performance Agent (`3-performance-tests.ts`)
-
-**Focus:** Latency, throughput, and scalability
-
-**Specialization:**
-- Benchmarks token counting (< 1ms target)
-- Measures route decision latency (< 5ms target)
-- Tests memory tree operations
-- Validates cost estimation accuracy
-- Measures batch routing throughput
-- Tests concurrent operation performance
-
-**Key Questions:**
-- Is token counting fast enough?
-- Can we route 100 queries per second?
-- Do memory operations scale?
-- Is cost estimation accurate?
-- Are operations deterministic?
-
-**Example Findings:**
-- Token counting too slow for real-time use
-- Memory tree operations don't scale
-- Inconsistent results under load
-- Cost calculation drift over time
-
-## Supporting Test Files
-
-### 4. Integration Tests (`4-integration-tests.ts`)
-
-**Focus:** Full pipeline workflows across components
-
-**Tests:**
-- Extract features → Route → Return structure
-- Memory tree add → search → verify
-- Cost tracker add → verify total
-- Provider register → get → deregister
-- Cross-component workflows
-
-### 5. Cost Model Tests (`5-cost-model-tests.ts`)
-
-**Focus:** Financial accuracy and budget enforcement
-
-**Tests:**
-- MODEL_COSTS matches actual provider pricing
-- estimateCost accuracy across tiers (free/cheap/mid/premium)
-- Cost comparison between models is correct
-- Budget enforcer respects limits
-- CostTracker calculations match expected values
-
-## How the Council Evaluates
-
-Each agent runs independently and reports:
-
-```
-Agent: Structure
-✅ All exports present
-✅ Provider configs valid
-❌ MODEL_PROFILES missing context_window field
-✅ Class instantiation works
-
-Result: 47/48 passed
-```
-
-The council **passes** only if **all agents pass**. This ensures:
-- No single aspect degrades
-- Issues in any dimension are caught
-- Overall system quality is maintained
-
-## Success Criteria
-
-For the council to pass, we require:
-
-| Agent | Metric | Target |
-|-------|--------|--------|
-| Structure | Tests passed | 100% |
-| Edge-Case | Tests passed | 100% |
-| Performance | Latency targets met | 100% |
-| Integration | Pipeline tests passed | 100% |
-| Cost Model | Cost accuracy | 100% |
-
-## Running the Council
-
-```bash
-# Run all tests
-node test-council/1-structure-tests.ts
-node test-council/2-edge-case-tests.ts
-node test-council/3-performance-tests.ts
-node test-council/4-integration-tests.ts
-node test-council/5-cost-model-tests.ts
-
-# Or use the test runner
-./test-council/run-all.sh
-```
-
-## Design Principles
-
-1. **Independence**: Each test file runs standalone
-2. **Isolation**: Tests don't depend on each other
-3. **Clear reporting**: Pass/fail with actionable messages
-4. **Focused scope**: Each agent has clear responsibilities
-5. **Realistic inputs**: Tests use real-world query patterns
-
-## Evolution
-
-The council pattern can be extended:
-
-- **Security Agent**: Input sanitization, injection attacks
-- **Reliability Agent**: Network failure simulation
-- **Scalability Agent**: Load testing with 1000+ concurrent users
-- **Compatibility Agent**: Node.js version, dependency conflicts
-
-## Credits
-
-Inspired by:
-- Ensemble methods in machine learning
-- Agent councils in production LLM systems (Mixtral, GPT-4)
-- Test-driven development principles
-- Separation of concerns in software architecture

package/tests/__mocks__/tokenUtils.ts

@@ -1,8 +0,0 @@
-// Mock for token utilities used by providerRetry
-export function countTokens(text: string): number {
-  return Math.ceil(text.length / 4);
-}
-
-export function estimateTokens(text: string): number {
-  return countTokens(text);
-}

package/tests/memory/episodicMemory.test.ts

@@ -1,227 +0,0 @@
-import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
-import { EpisodicMemoryStore } from '../../tmlpd-pi-extension/src/memory/episodicMemory';
-
-describe('EpisodicMemoryStore', () => {
-  let store: EpisodicMemoryStore;
-
-  const sampleEntry = {
-    task: {
-      description: 'Test task execution',
-      type: 'code_generation',
-      complexity: 3,
-    },
-    result: {
-      success: true,
-      output: 'Successfully generated code',
-      duration_ms: 1500,
-    },
-    agent: {
-      id: 'agent-1',
-      model: 'gpt-4o',
-      provider: 'openai',
-    },
-    metadata: { source: 'test' },
-    importance: 5,
-  };
-
-  beforeEach(() => {
-    store = new EpisodicMemoryStore(100);
-  });
-
-  describe('store and query', () => {
-    it('stores an entry and returns a non-empty id', () => {
-      const id = store.store(sampleEntry);
-      expect(id).toBeTruthy();
-      expect(typeof id).toBe('string');
-      expect(id.length).toBeGreaterThan(0);
-    });
-
-    it('stores multiple entries and retrieves them', () => {
-      store.store(sampleEntry);
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Second task' } });
-
-      const results = store.query({ limit: 10 });
-      expect(results.length).toBe(2);
-    });
-
-    it('filters by task type', () => {
-      store.store(sampleEntry);
-      store.store({
-        ...sampleEntry,
-        task: { ...sampleEntry.task, type: 'data_analysis' },
-      });
-
-      const codeResults = store.query({ task_type: 'code_generation' });
-      expect(codeResults.length).toBe(1);
-      expect(codeResults[0].task.type).toBe('code_generation');
-
-      const analysisResults = store.query({ task_type: 'data_analysis' });
-      expect(analysisResults.length).toBe(1);
-    });
-
-    it('filters by keywords', () => {
-      store.store(sampleEntry);
-      store.store({
-        ...sampleEntry,
-        task: { ...sampleEntry.task, description: 'Database query optimization' },
-      });
-
-      const results = store.query({ keywords: ['database'] });
-      expect(results.length).toBe(1);
-      expect(results[0].task.description).toContain('Database');
-    });
-
-    it('returns newest entries first', () => {
-      store.store(sampleEntry);
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Second' } });
-
-      const results = store.query({ limit: 10 });
-      expect(results[results.length - 1].task.description).toBe('Second');
-    });
-
-    it('respects limit parameter', () => {
-      for (let i = 0; i < 10; i++) {
-        store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: `Task ${i}` } });
-      }
-
-      expect(store.query({ limit: 3 }).length).toBe(3);
-      expect(store.query({ limit: 20 }).length).toBe(10);
-    });
-  });
-
-  describe('max entries eviction', () => {
-    it('evicts oldest entries when at capacity', () => {
-      const smallStore = new EpisodicMemoryStore(3);
-      smallStore.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'A' } });
-      smallStore.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'B' } });
-      smallStore.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'C' } });
-      smallStore.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'D' } });
-
-      expect(smallStore.query({ limit: 10 }).length).toBe(3);
-      const descriptions = smallStore.query({ limit: 10 }).map(e => e.task.description);
-      expect(descriptions).not.toContain('A');
-      expect(descriptions).toContain('D');
-    });
-  });
-
-  describe('getSimilarTasks', () => {
-    it('finds tasks with overlapping words', () => {
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Build REST API endpoint' } });
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Deploy database cluster' } });
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Fix API authentication bug' } });
-
-      const similar = store.getSimilarTasks('Build API endpoint');
-      expect(similar.length).toBeGreaterThanOrEqual(1);
-      // "API" should match entries 0 and 2
-      const descriptions = similar.map(e => e.task.description);
-      expect(descriptions.some(d => d.includes('API'))).toBe(true);
-    });
-
-    it('returns empty array for no matches', () => {
-      store.store(sampleEntry);
-      const similar = store.getSimilarTasks('xyznonexistent12345');
-      expect(similar.length).toBe(0);
-    });
-
-    it('respects limit parameter', () => {
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Task A' } });
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Task B' } });
-      store.store({ ...sampleEntry, task: { ...sampleEntry.task, description: 'Task C' } });
-
-      const similar = store.getSimilarTasks('Task', 2);
-      expect(similar.length).toBeLessThanOrEqual(2);
-    });
-  });
-
-  describe('getStats', () => {
-    it('returns zeroed stats for empty store', () => {
-      const stats = store.getStats();
-      expect(stats.total_entries).toBe(0);
-      expect(stats.indexed_keywords).toBe(0);
-      expect(stats.success_rate).toBe(0);
-      expect(stats.avg_duration_ms).toBe(0);
-    });
-
-    it('returns accurate stats after entries', () => {
-      store.store(sampleEntry);
-      store.store({
-        ...sampleEntry,
-        result: { ...sampleEntry.result, success: false },
-      });
-
-      const stats = store.getStats();
-      expect(stats.total_entries).toBe(2);
-      expect(stats.success_rate).toBe(0.5);
-      expect(stats.avg_duration_ms).toBe(1500);
-    });
-  });
-
-  describe('clear', () => {
-    it('removes all entries', () => {
-      store.store(sampleEntry);
-      store.store(sampleEntry);
-      expect(store.query({ limit: 10 }).length).toBe(2);
-
-      store.clear();
-      expect(store.query({ limit: 10 }).length).toBe(0);
-      expect(store.getStats().total_entries).toBe(0);
-    });
-  });
-
-  describe('persistence', () => {
-    let fsMock: any;
-
-    beforeEach(() => {
-      fsMock = {
-        existsSync: vi.fn().mockReturnValue(false),
-        readFileSync: vi.fn(),
-        writeFileSync: vi.fn(),
-        unlinkSync: vi.fn(),
-      };
-
-      vi.doMock('fs', () => fsMock);
-    });
-
-    afterEach(() => {
-      vi.restoreAllMocks();
-    });
-
-    it('loadFromDisk returns false when file does not exist', () => {
-      const store2 = new EpisodicMemoryStore(100, '/tmp/test-memory.json');
-      expect(store2['loadFromDisk']()).toBe(false);
-    });
-
-    it('saveToDisk returns false when no persistence path', () => {
-      expect(store.saveToDisk()).toBe(false);
-    });
-
-    it('autoPersist triggers after every 3 entries', () => {
-      const saveSpy = vi.spyOn(store as any, 'autoPersist');
-      store.store(sampleEntry);
-      store.store(sampleEntry);
-      expect(saveSpy).toHaveBeenCalledTimes(2); // called on each store
-      expect((store as any).entries.length % 3 === 0).toBe(false);
-
-      store.store(sampleEntry); // 3rd entry
-      expect((store as any).entries.length % 3 === 0).toBe(true);
-    });
-  });
-
-  describe('keyword indexing', () => {
-    it('indexes words longer than 3 characters', () => {
-      store.store(sampleEntry);
-      const index = (store as any).keywordIndex as Map<string, string[]>;
-
-      // 'test' (4 chars) should be indexed, 'is' (2 chars) should not
-      const keys = Array.from(index.keys());
-      expect(keys.some(k => k.length <= 3)).toBe(false);
-    });
-
-    it('automatically indexes on store', () => {
-      store.store(sampleEntry);
-      const index = (store as any).keywordIndex as Map<string, string[]>;
-      expect(index.has('test')).toBe(true);
-      expect(index.has('task')).toBe(true);
-    });
-  });
-});