quay-ai-factory 0.2.0 → 0.3.1

package/README.md

@@ -1,18 +1,18 @@
 # Quay - Autonomous AI Software Factory
 
-> Self-hosted AI agents with MCP integration, customizable pipelines, real-time Mission Control dashboard, cost transparency, and full audit trails.
+> Self-hosted AI agents with MCP integration, A3M routing, customizable pipelines, real-time Mission Control dashboard, cost transparency, and full audit trails.
 
 ## 🚀 Features
 
 ### 5 AI-Powered Features (Built on A3M Router)
 
-| Feature | Description |
-|---------|-------------|
-| **Human-in-Loop Verification** | Require approval before critical actions (deploy, delete) |
-| **Determinism & Replay** | Black-box recorder captures every step for reproducibility |
-| **Visual Pipeline Builder** | Drag-drop YAML pipeline generation with built-in templates |
-| **Agent RAG** | Knowledge grounding to reduce hallucinations |
-| **Self-Healing Agents** | Auto-detect and recover from failures |
+| Feature | Description | Agent Integration |
+|---------|-------------|-------------------|
+| **Human-in-Loop Verification** | Require approval before critical actions | Auto-triggers on deploy, delete, rm, etc. |
+| **Determinism & Replay** | Black-box recorder captures every step | Snapshots after each LLM call |
+| **Visual Pipeline Builder** | Drag-drop YAML pipeline generation | Built-in templates for common workflows |
+| **Agent RAG** | Knowledge grounding to reduce hallucinations | Applied before first LLM call |
+| **Self-Healing Agents** | Auto-detect and recover from failures | Detects 7 failure types, 5 recovery strategies |
 
 ### Core Capabilities
 
@@ -25,7 +25,7 @@
 ## 📦 Installation
 
 ```bash
-npm install
+npm install quay-ai-factory
 ```
 
 ## 🏃‍♂️ Running
@@ -42,6 +42,88 @@ node build
 bun test
 ```
 
+## 🤖 Agent Runner
+
+The `AgentRunner` orchestrates the observe→plan→act loop with all 5 AI features:
+
+```typescript
+import { agentRunner } from './server/agents/agentRunner.js';
+
+// Run an agent with all features enabled
+const result = await agentRunner.run(
+  { id: 'task-1', title: 'Deploy to production', description: '...' },
+  { id: 'agent-1', name: 'DeployBot', type: 'coder', provider: 'openai', model: 'gpt-4o', config: '{}' },
+  'run-123',
+  'You are a deployment agent...',
+  ['bash', 'deploy'],
+  {
+    knowledgeBaseId: 'kb-1',        // Enable RAG grounding
+    verificationConfig: {            // Custom verification rules
+      criticalActions: ['deploy', 'delete'],
+      autoApproveBelow: 0.10,
+    },
+    maxSteps: 30,                   // Custom step limit
+  }
+);
+```
+
+### How Features Integrate
+
+```
+User Request
+    ↓
+AgentRunner.run()
+    ↓
+┌─────────────────────────────────────────┐
+│ 1. RAG Grounding                        │
+│    buildGroundedPrompt()                │
+│    → Adds relevant knowledge to prompt  │
+└─────────────────────────────────────────┘
+    ↓
+┌─────────────────────────────────────────┐
+│ 2. LLM Call                             │
+│    callLLM() with A3M routing           │
+└─────────────────────────────────────────┘
+    ↓
+┌─────────────────────────────────────────┐
+│ 3. Self-Healing Check                   │
+│    detectFailure() → getRecoveryStrategy│
+└─────────────────────────────────────────┘
+    ↓
+┌─────────────────────────────────────────┐
+│ 4. Determinism Snapshot                 │
+│    createSnapshot() with state hash     │
+└─────────────────────────────────────────┘
+    ↓
+┌─────────────────────────────────────────┐
+│ 5. Health Update                        │
+│    updateHealthStatus()                 │
+└─────────────────────────────────────────┘
+    ↓
+┌─────────────────────────────────────────┐
+│ 6. Critical Action Verification        │
+│    createVerificationRequest()          │
+│    → Pauses if approval needed         │
+└─────────────────────────────────────────┘
+    ↓
+Tool Execution → Loop
+```
+
+## 🔧 Configuration
+
+```bash
+# Environment variables
+OPENAI_API_KEY=sk-...                    # OpenAI API key
+ANTHROPIC_API_KEY=sk-ant-...             # Anthropic API key
+QUAY_DEFAULT_LLM_URL=https://api.openai.com/v1  # Default LLM endpoint
+
+EMBEDDING_PROVIDER=openai                # openai, local, or mock
+EMBEDDING_ENDPOINT=http://localhost:11434/api/embeddings
+QUAY_DB_PATH=./quay.db                   # Database path
+SSE_ENABLED=true                         # Enable SSE
+NODE_ENV=production                      # production, development, test
+```
+
 ## 🧪 Features
 
 ### Verification Dashboard (`/verification`)
@@ -64,6 +146,8 @@ const request = await createVerificationRequest(
 approveRequest(request.id, 'admin');
 ```
 
+**Critical Actions** (auto-detected): `deploy`, `delete`, `rm`, `drop`, `truncate`, `shutdown`, `restart`
+
 ### Health Monitor (`/health`)
 Real-time agent health status with self-healing.
 
@@ -76,8 +160,13 @@ updateHealthStatus('agent-1', { latencyMs: 150, success: true });
 // Detect failure and get recovery strategy
 const failure = detectFailure(new Error('timeout'));
 const strategy = getRecoveryStrategy(failure.type);
+// → { action: 'retry', maxRetries: 3, fallbackModel: 'gpt-4o-mini' }
 ```
 
+**Failure Types**: `timeout`, `rate_limit`, `api_error`, `context_overflow`, `tool_error`, `unknown`
+
+**Recovery Strategies**: `retry`, `fallback_model`, `fallback_agent`, `abort`, `skip_step`
+
 ### Knowledge Base (`/knowledge`)
 RAG-powered knowledge grounding.
 
@@ -103,6 +192,8 @@ const templates = getTemplates('code-review');
 const yaml = generateYAML(nodes, edges);
 ```
 
+**Built-in Templates**: `code-review`, `security-scan`, `data-processing`, `research-agent`
+
 ### Replay (`/replay`)
 Deterministic replay of agent runs.
 
@@ -113,43 +204,34 @@ await createSnapshot(runId, stepIndex, state);
 const session = await startReplay(runId);
 ```
 
-## 🔧 Configuration
-
-```bash
-# Environment variables
-OPENAI_API_KEY=sk-...          # OpenAI API key
-EMBEDDING_PROVIDER=openai       # openai, local, or mock
-EMBEDDING_ENDPOINT=http://localhost:11434/api/embeddings
-QUAY_DB_PATH=./quay.db          # Database path
-SSE_ENABLED=true                # Enable SSE
-NODE_ENV=production             # production, development, test
-```
-
 ## 📁 Project Structure
 
 ```
 quay/
 ├── src/
 │   ├── lib/
-│   │   ├── features/          # 5 AI features
-│   │   │   ├── verification.ts
-│   │   │   ├── determinism.ts
-│   │   │   ├── visualBuilder.ts
-│   │   │   ├── agentRAG.ts
-│   │   │   └── selfHealing.ts
-│   │   ├── components/         # Svelte UI components
-│   │   ├── sse/              # SSE broadcaster
-│   │   ├── a3m.ts            # A3M router integration
-│   │   ├── db.ts             # Database layer
-│   │   └── embedding.ts       # Embedding service
+│   │   ├── features/              # 5 AI features
+│   │   │   ├── verification.ts    # Human-in-loop
+│   │   │   ├── determinism.ts     # Snapshots & replay
+│   │   │   ├── visualBuilder.ts   # Pipeline builder
+│   │   │   ├── agentRAG.ts        # Knowledge grounding
+│   │   │   └── selfHealing.ts     # Auto-recovery
+│   │   ├── components/            # Svelte UI components
+│   │   ├── sse/                  # SSE broadcaster
+│   │   ├── a3m.ts                 # A3M router integration
+│   │   ├── db.ts                  # Database layer
+│   │   └── embedding.ts           # Embedding service
 │   ├── routes/
-│   │   ├── api/              # API endpoints
-│   │   └── [page].svelte     # UI pages
+│   │   ├── api/                   # API endpoints
+│   │   └── [page].svelte          # UI pages
 │   └── server/
-│       ├── db/                # Database schema
-│       └── a3m/               # A3M config
+│       ├── agents/
+│       │   └── agentRunner.ts     # Main agent loop (integrates all features)
+│       ├── db/                    # Database schema
+│       ├── features/              # Feature implementations
+│       └── ...
 └── tests/
-    └── __tests__/             # Integration tests
+    └── __tests__/                 # Integration tests
 ```
 
 ## 🧪 Testing
@@ -172,7 +254,6 @@ Total: 56 tests passing
 | Endpoint | Description |
 |----------|-------------|
 | `GET /api/verification` | List verification requests |
-| `POST /api/verification` | Create verification request |
 | `POST /api/verification/[id]/approve` | Approve request |
 | `POST /api/verification/[id]/reject` | Reject request |
 | `GET /api/agent-health` | Get all agent health status |
@@ -188,7 +269,7 @@ Total: 56 tests passing
 
 | Route | Description |
 |-------|-------------|
-| `/` | Home |
+| `/` | Home / Mission Control |
 | `/features` | Features dashboard |
 | `/verification` | Verification dashboard |
 | `/health` | Agent health monitor |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "quay-ai-factory",
-  "version": "0.2.0",
+  "version": "0.3.1",
   "description": "Quay — Autonomous AI Software Factory. Self-hosted AI agents, MCP integration, customizable pipelines, real-time Mission Control dashboard, cost transparency, and full audit trails.",
   "keywords": [
     "ai-software-factory",

package/src/server/agents/agentRunner.ts

@@ -12,6 +12,35 @@ import { mcpRegistry } from '../mcp/index.js';
 import { memoryTree } from '../memory/memoryTree.js';
 import type { MCPToolResult } from '../mcp/index.js';
 
+// ----------------------------------------------------------------
+// Import 5 AI Features
+// ----------------------------------------------------------------
+import {
+  createVerificationRequest,
+  approveRequest,
+  rejectRequest,
+  type VerificationConfig,
+} from '../features/verification.js';
+
+import {
+  createSnapshot,
+  verifyDeterminism,
+} from '../features/determinism.js';
+
+import {
+  updateHealthStatus,
+  detectFailure,
+  getRecoveryStrategy,
+  healthStatus,
+  type AgentHealth,
+} from '../features/selfHealing.js';
+
+import {
+  buildGroundedPrompt,
+  knowledgeBases,
+  type KnowledgeBaseConfig,
+} from '../features/agentRAG.js';
+
 // ----------------------------------------------------------------
 // Agent LLM Interface (pluggable — uses A3M or direct provider)
 // ----------------------------------------------------------------
@@ -38,7 +67,6 @@ async function callLLM(
   config: LLMConfig,
   maxTokens = 4096
 ): Promise<LLMResponse> {
-  // baseURL must come from agent config — no magic URL construction
   if (!config.baseURL) {
     throw new Error(`No baseURL configured for provider ${config.provider}. Set baseURL in agent config or QUAY_DEFAULT_LLM_URL env var.`);
   }
@@ -69,7 +97,6 @@ async function callLLM(
   };
   const content = data.choices[0]?.message?.content ?? '';
 
-  // Parse tool calls from <tool_call> XML tags
   const toolCalls: Array<{ name: string; arguments: Record<string, unknown> }> = [];
   const toolRegex = /<tool_call>\s*<name>([\w:]+)<\/name>\s*<args>([\s\S]*?)<\/args>\s*<\/tool_call>/g;
   let match;
@@ -93,8 +120,7 @@ async function callLLM(
 }
 
 // ----------------------------------------------------------------
-// Sandbox Execution (Docker via HTTP)
-// Looks up sandbox endpoint from DB instead of using env var
+// Sandbox Execution
 // ----------------------------------------------------------------
 
 interface SandboxExecResult {
@@ -104,14 +130,13 @@ interface SandboxExecResult {
 }
 
 async function execInSandbox(sandboxId: string, command: string): Promise<SandboxExecResult> {
-  // Look up sandbox endpoint from DB — do NOT use hardcoded SANDBOX_URL
   const [sandbox] = dbq.select<{ id: string; endpoint: string | null; status: string }>('sandboxes', { id: sandboxId });
   if (!sandbox) throw new Error(`Sandbox not found: ${sandboxId}`);
   if (!sandbox.endpoint) throw new Error(`Sandbox endpoint not set: ${sandboxId}`);
 
   const res = await fetch(`${sandbox.endpoint}/exec`, {
     method: 'POST',
-    headers: { 'Content-Type': 'application/json' },
+    headers: { ' Content-Type': 'application/json' },
     body: JSON.stringify({ command, timeout: 300 }),
   });
   if (!res.ok) throw new Error(`Sandbox exec failed: ${res.status} ${await res.text()}`);
@@ -119,33 +144,27 @@ async function execInSandbox(sandboxId: string, command: string): Promise<Sandbo
 }
 
 // ----------------------------------------------------------------
-// Model Pricing for Cost Tracking
+// Model Pricing
 // ----------------------------------------------------------------
 
 const MODEL_PRICING: Record<string, { input: number; output: number }> = {
-  // Anthropic
   'claude-3-5-sonnet-latest': { input: 3, output: 15 },
   'claude-sonnet-4-20250514': { input: 3, output: 15 },
   'claude-3-5-haiku-latest': { input: 0.8, output: 4 },
-  // OpenAI
   'gpt-4o': { input: 2.5, output: 10 },
   'gpt-4o-mini': { input: 0.15, output: 0.6 },
   'gpt-4-turbo': { input: 10, output: 30 },
   'gpt-4': { input: 30, output: 60 },
-  // Groq
   'llama-3.3-70b-versatile': { input: 0.2, output: 0.2 },
   'llama-3.1-8b-instant': { input: 0.05, output: 0.08 },
   'mixtral-8x7b-32768': { input: 0.24, output: 0.24 },
-  // Ollama (local — free)
   'llama3': { input: 0, output: 0 },
   'llama3.1': { input: 0, output: 0 },
   'codellama': { input: 0, output: 0 },
   'qwen2.5-coder': { input: 0, output: 0 },
   'qwen2.5': { input: 0, output: 0 },
-  // Google
   'gemini-2.5-flash': { input: 0.1, output: 0.4 },
   'gemini-1.5-pro': { input: 1.25, output: 5 },
-  // Zhipu
   'glm-4': { input: 0.1, output: 0.3 },
   'glm-4-flash': { input: 0.0, output: 0.0 },
   'glm-5': { input: 0.5, output: 1.5 },
@@ -161,24 +180,48 @@ function calculateCost(model: string, tokensIn: number, tokensOut: number): numb
 }
 
 // ----------------------------------------------------------------
-// Agent Runner
+// Agent Runner with 5 AI Features Integrated
 // ----------------------------------------------------------------
 
 export class AgentRunner {
   maxSteps = 20;
 
+  // Feature configs (can be set per-agent or globally)
+  verificationConfig: VerificationConfig = {
+    enabled: true,
+    approvalThreshold: 'critical',
+    criticalActions: ['deploy', 'delete', 'rm', 'drop', 'truncate', 'shutdown', 'restart'],
+    autoApproveBelow: 0.05,
+    useA3MRouting: true,
+  };
+
+  knowledgeBaseConfig: KnowledgeBaseConfig = {
+    defaultThreshold: 0.7,
+    maxChunks: 5,
+    embeddingModel: 'text-embedding-3-small',
+  };
+
   async run(
     task: { id: string; title: string; description?: string },
     agent: { id: string; name: string; type: string; provider: string; model: string; config: string },
     runId: string,
     instructions: string,
     allowedTools: string[],
+    options?: {
+      knowledgeBaseId?: string;
+      verificationConfig?: Partial<VerificationConfig>;
+      maxSteps?: number;
+    },
   ): Promise<{ success: boolean; error?: string; cost: number; tokensIn: number; tokensOut: number }> {
     const startTime = Date.now();
     let totalCost = 0;
     let totalTokensIn = 0;
     let totalTokensOut = 0;
     let stepIndex = 0;
+    const maxSteps = options?.maxSteps ?? this.maxSteps;
+
+    // Merge verification config
+    const verificationConfig = { ...this.verificationConfig, ...options?.verificationConfig };
 
     // Parse agent config for baseURL and temperature
     let agentConfig: Record<string, unknown> = {};
@@ -188,10 +231,23 @@ export class AgentRunner {
 
     let currentPrompt = `Task: ${task.title}\n\n${task.description ?? ''}`;
 
-    while (stepIndex < this.maxSteps) {
+    // Apply RAG grounding if knowledge base is configured
+    if (options?.knowledgeBaseId) {
+      currentPrompt = await buildGroundedPrompt(
+        options.knowledgeBaseId,
+        currentPrompt,
+        this.knowledgeBaseConfig
+      );
+    }
+
+    // Initialize health status for this agent
+    updateHealthStatus(agent.id, { success: true, latencyMs: 0 });
+
+    while (stepIndex < maxSteps) {
       let llmResponse: LLMResponse;
+      const stepStartTime = Date.now();
+
       try {
-        // baseURL from agent config (required) — no magic URL construction
         const config: LLMConfig = {
           provider: agent.provider,
           model: agent.model,
@@ -202,7 +258,23 @@ export class AgentRunner {
         };
         llmResponse = await callLLM(currentPrompt, systemPrompt, config);
       } catch (e) {
+        // Self-healing: detect failure and get recovery strategy
+        const failure = detectFailure(e as Error);
+        const strategy = getRecoveryStrategy(failure.type);
+
+        updateHealthStatus(agent.id, { error: true });
         this.recordFailureSync(runId, stepIndex, String(e));
+
+        // Apply recovery strategy if available
+        if (strategy) {
+          sseBroadcaster.broadcast('agent:recovery', {
+            agentId: agent.id,
+            failure: failure.type,
+            strategy: strategy.action,
+            runId,
+          });
+        }
+
         return { success: false, error: String(e), cost: totalCost, tokensIn: totalTokensIn, tokensOut: totalTokensOut };
       }
 
@@ -213,15 +285,35 @@ export class AgentRunner {
         totalCost += calculateCost(agent.model, llmResponse.usage.inputTokens, llmResponse.usage.outputTokens);
       }
 
-      const stepId = uuidv4();
-      dbq.insert('step_attempts', {
-        id: stepId, run_id: runId, step_index: stepIndex,
-        thought: llmResponse.thought, action: null, action_args: '{}',
-        observation: null, error: null, started_at: Date.now(), completed_at: null,
+      // Update health with latency
+      const stepLatency = Date.now() - stepStartTime;
+      updateHealthStatus(agent.id, { success: true, latencyMs: stepLatency });
+
+      // Create deterministic snapshot for this step
+      const snapshot = createSnapshot(runId, stepIndex, {
+        prompt: currentPrompt,
+        context: {
+          taskId: task.id,
+          agentId: agent.id,
+          knowledgeBaseId: options?.knowledgeBaseId,
+        },
+        memory: memoryTree.getRecentMemories(runId),
+        thought: llmResponse.thought,
+        toolCalls: llmResponse.toolCalls,
+        tokensIn: llmResponse.usage?.inputTokens ?? 0,
+        tokensOut: llmResponse.usage?.outputTokens ?? 0,
+        costUSD: totalCost,
+        latencyMs: stepLatency,
       });
 
-      memoryTree.pushStep(runId, llmResponse.thought, '', '');
-      sseBroadcaster.broadcast('run:progress', { runId, stepIndex, thought: llmResponse.thought });
+      // Broadcast step progress with snapshot info
+      sseBroadcaster.broadcast('run:progress', {
+        runId,
+        stepIndex,
+        thought: llmResponse.thought,
+        snapshotId: snapshot.id,
+        determinismHash: snapshot.stateHash,
+      });
 
       if (llmResponse.isDone) {
         dbq.update('runs', {
@@ -232,16 +324,95 @@ export class AgentRunner {
           tokens_out: totalTokensOut,
         }, { id: runId });
         await memoryTree.updateAgentLongTerm(agent.id, { id: runId } as any, true);
-        sseBroadcaster.broadcast('run:completed', { runId, cost: totalCost, durationMs: Date.now() - startTime });
+        sseBroadcaster.broadcast('run:completed', {
+          runId,
+          cost: totalCost,
+          durationMs: Date.now() - startTime,
+          totalSteps: stepIndex + 1,
+        });
         return { success: true, cost: totalCost, tokensIn: totalTokensIn, tokensOut: totalTokensOut };
       }
 
+      // Execute tool calls with verification for critical actions
       for (const tc of llmResponse.toolCalls) {
+        const toolName = tc.name.toLowerCase();
+        const isCritical = verificationConfig.criticalActions.some(
+          action => toolName.includes(action.toLowerCase())
+        );
+
+        // Human-in-Loop Verification for critical actions
+        if (isCritical && verificationConfig.enabled) {
+          const estimatedCost = totalCost * 0.1; // Estimate 10% of current cost
+
+          const verificationReq = await createVerificationRequest(
+            agent.id,
+            task.id,
+            tc.name,
+            `Executing ${tc.name} with args: ${JSON.stringify(tc.arguments)}`,
+            estimatedCost,
+            verificationConfig
+          );
+
+          if (verificationReq) {
+            // Broadcast verification request
+            sseBroadcaster.broadcast('verification:pending', {
+              requestId: verificationReq.id,
+              agentId: agent.id,
+              runId,
+              action: tc.name,
+              args: tc.arguments,
+            });
+
+            // For now, auto-reject if pending (agent should wait for human approval in real scenario)
+            // In production, this would be an async wait for human approval
+            sseBroadcaster.broadcast('agent:paused', {
+              runId,
+              stepIndex,
+              reason: 'verification_pending',
+              requestId: verificationReq.id,
+            });
+
+            // Apply recovery strategy - skip this step and continue
+            const failure = detectFailure(new Error('verification_required'));
+            const strategy = getRecoveryStrategy(failure.type);
+
+            currentPrompt += `\n\n[${tc.name}] → VERIFICATION REQUIRED: Action paused for human approval. Request ID: ${verificationReq.id}`;
+            continue;
+          }
+        }
+
         const toolResult = await this.executeTool(tc.name, tc.arguments);
-        currentPrompt += `\n\n[${tc.name}] → ${JSON.stringify(toolResult)}`;
+        const resultStr = JSON.stringify(toolResult);
+        currentPrompt += `\n\n[${tc.name}] → ${resultStr}`;
+
+        // Update snapshot with observation
+        snapshot.context = {
+          ...snapshot.context,
+          lastObservation: resultStr,
+          lastToolCall: tc.name,
+        };
+
+        // Check if tool execution failed
+        if (!toolResult.success && toolResult.error) {
+          updateHealthStatus(agent.id, { error: true });
+
+          const failure = detectFailure(new Error(toolResult.error));
+          const strategy = getRecoveryStrategy(failure.type);
+
+          if (strategy) {
+            sseBroadcaster.broadcast('agent:recovery', {
+              agentId: agent.id,
+              failure: failure.type,
+              strategy: strategy.action,
+              runId,
+              stepIndex,
+            });
+          }
+        }
+
         dbq.update('step_attempts', {
-          action: tc.name, action_args: JSON.stringify(tc.arguments), observation: JSON.stringify(toolResult)
-        }, { id: stepId });
+          action: tc.name, action_args: JSON.stringify(tc.arguments), observation: resultStr
+        }, { id: snapshot.id });
       }
 
       stepIndex++;
@@ -249,6 +420,7 @@ export class AgentRunner {
 
     const err = 'Max steps exceeded';
     this.recordFailureSync(runId, stepIndex, err);
+    updateHealthStatus(agent.id, { error: true });
     return { success: false, error: err, cost: totalCost, tokensIn: totalTokensIn, tokensOut: totalTokensOut };
   }
 
@@ -267,7 +439,6 @@ export class AgentRunner {
         return { tool: name, success: false, error: String(e) };
       }
     }
-    // Route through MCP
     return mcpRegistry.routeTool(name, args);
   }