snow-flow 2.9.9 → 3.0.0

package/src/agents/README.md

@@ -0,0 +1,192 @@
+# ServiceNow Specialist Agents
+
+This directory contains the implementation of ServiceNow specialist agents that work together through Claude Code to build, deploy, and maintain ServiceNow solutions.
+
+## ⚠️ IMPORTANT: Flow Builder Agent Removed in v1.4.0 ⚠️
+
+**Note**: The Flow Builder Agent and all flow-related functionality have been removed in v1.4.0 due to critical bugs. The agent file may still exist but is no longer functional. Please use ServiceNow's native Flow Designer interface directly for flow creation.
+
+## 🤖 Available Agents
+
+### 1. Widget Creator Agent (`widget-creator-agent.ts`)
+Specializes in creating ServiceNow Service Portal widgets.
+
+**Capabilities:**
+- HTML template creation
+- CSS styling and responsive design
+- Client-side JavaScript development
+- Server-side data processing
+- Chart.js integration
+- Demo data generation
+
+**MCP Tools Used:**
+- `snow_deploy` - Deploy widgets to ServiceNow
+- `snow_preview_widget` - Preview widget rendering
+- `snow_widget_test` - Test widget functionality
+
+### 2. Flow Builder Agent (`flow-builder-agent.ts`)
+Specializes in creating ServiceNow Flow Designer workflows.
+
+**Capabilities:**
+- Business process design
+- Flow trigger configuration
+- Approval workflow creation
+- Integration flow building
+- Error handling design
+
+**MCP Tools Used:**
+- `snow_create_flow` - Create flows from natural language
+- `snow_test_flow_with_mock` - Test flows with mock data
+- `snow_link_catalog_to_flow` - Link flows to catalog items
+
+### 3. Script Writer Agent (`script-writer-agent.ts`)
+Specializes in creating ServiceNow scripts.
+
+**Capabilities:**
+- Business rule creation
+- Script include development
+- Client script implementation
+- Scheduled job scripts
+- Performance optimization
+
+**MCP Tools Used:**
+- `snow_create_script_include` - Create script includes
+- `snow_create_business_rule` - Create business rules
+- `snow_create_client_script` - Create client scripts
+
+### 4. Test Agent (`test-agent.ts`)
+Specializes in testing ServiceNow artifacts.
+
+**Capabilities:**
+- Test scenario creation
+- Mock data generation
+- Integration testing
+- Performance validation
+- Quality assurance
+
+**MCP Tools Used:**
+- `snow_test_flow_with_mock` - Test flows
+- `snow_widget_test` - Test widgets
+- `snow_comprehensive_flow_test` - Comprehensive testing
+- `snow_cleanup_test_artifacts` - Clean up test data
+
+### 5. Security Agent (`security-agent.ts`)
+Specializes in ServiceNow security and compliance.
+
+**Capabilities:**
+- Security policy enforcement
+- Vulnerability scanning
+- Access control validation
+- Compliance checking (SOX, GDPR, HIPAA)
+- Security best practices
+
+**MCP Tools Used:**
+- `snow_create_access_control` - Create ACLs
+- `snow_security_scan` - Security scanning
+- `snow_run_compliance_scan` - Compliance validation
+
+## 🏗️ Architecture
+
+### Base Agent Class (`base-agent.ts`)
+All agents extend the `BaseAgent` class which provides:
+- Shared memory integration via SQLite
+- Progress reporting to Queen Agent
+- Inter-agent communication
+- Error handling and logging
+- Lifecycle management
+
+### Agent Communication Pattern
+```typescript
+// Agents communicate through shared memory
+await agent.storeArtifact(artifact);
+await agent.reportProgress('Task completed', 100);
+await agent.sendMessage(otherAgentId, 'coordination', data);
+```
+
+### Integration with Queen Agent
+The Queen Agent uses `AgentFactory` to spawn these specialized agents:
+```typescript
+const agent = await agentFactory.createSpecializedAgent('widget-creator');
+const result = await agent.execute(instruction, context);
+```
+
+## 🚀 Usage Examples
+
+### Creating a Widget
+```typescript
+const widgetAgent = new WidgetCreatorAgent({ debugMode: true });
+const result = await widgetAgent.execute(
+  'Create an incident dashboard widget with real-time charts'
+);
+```
+
+### Building a Flow
+```typescript
+const flowAgent = new FlowBuilderAgent({ debugMode: true });
+const result = await flowAgent.execute(
+  'Create an approval flow for catalog requests with manager approval'
+);
+```
+
+### Writing a Script
+```typescript
+const scriptAgent = new ScriptWriterAgent({ debugMode: true });
+const result = await scriptAgent.execute(
+  'Create a business rule to calculate incident priority'
+);
+```
+
+### Testing Artifacts
+```typescript
+const testAgent = new TestAgent({ debugMode: true });
+const result = await testAgent.execute(
+  'Test the incident dashboard widget with comprehensive scenarios'
+);
+```
+
+### Security Scanning
+```typescript
+const securityAgent = new SecurityAgent({ debugMode: true });
+const result = await securityAgent.execute(
+  'Perform security scan on all recent artifacts'
+);
+```
+
+## 🧪 Testing
+
+Run the agent test suite:
+```bash
+npm test src/agents/test-agents.ts
+```
+
+## 📝 Best Practices
+
+1. **Always use shared memory** for coordination between agents
+2. **Report progress regularly** for long-running tasks
+3. **Handle errors gracefully** and provide meaningful error messages
+4. **Store artifacts** in shared memory for other agents to use
+5. **Clean up resources** when agent tasks complete
+
+## 🔄 Agent Lifecycle
+
+1. **Initialization**: Agent is created with configuration
+2. **Execution**: Agent performs its specialized task
+3. **Coordination**: Agent communicates with other agents
+4. **Completion**: Agent reports results and stores artifacts
+5. **Cleanup**: Agent releases resources
+
+## 🔗 MCP Tool Integration
+
+Each agent knows which MCP tools to recommend for its tasks. The actual MCP tool execution happens through Claude Code, not directly by the agents. Agents provide:
+- Tool recommendations
+- Parameter preparation
+- Result interpretation
+- Error handling
+
+## 🎯 Future Enhancements
+
+- [ ] Add more specialized agents (Catalog Manager, Integration Specialist)
+- [ ] Implement agent learning from past executions
+- [ ] Add performance metrics collection
+- [ ] Enhance inter-agent coordination protocols
+- [ ] Add agent health monitoring

package/src/health/README.md

@@ -0,0 +1,161 @@
+# Snow-Flow System Health Monitoring
+
+## Overview
+
+The System Health module provides comprehensive real-time monitoring of all Snow-Flow components, replacing placeholder metrics with actual system measurements.
+
+## Real Metrics Implemented
+
+### 1. **CPU Usage** (Real-time)
+- Uses CPU time differentials for accurate usage calculation
+- Supports multi-core systems
+- Automatically averages across all CPU cores
+- Updates every 100ms for initial reading
+
+### 2. **Memory Usage** (Real-time)
+- **System Memory**: Total and used memory from OS
+- **Process Memory**: 
+  - Heap usage (used/total)
+  - RSS (Resident Set Size)
+  - External memory
+  - Array buffers
+- Percentage calculations for threshold monitoring
+
+### 3. **Disk Usage** (Real-time)
+- **Cross-platform support**:
+  - Unix/macOS: Uses `df` command
+  - Windows: Uses `wmic` command
+- Monitors the filesystem containing the application
+- Returns usage percentage and total size in GB
+- Handles edge cases (long filesystem names, parsing errors)
+
+### 4. **Network Connectivity**
+- DNS lookup test to verify internet connectivity
+- Used for ServiceNow API health checks
+
+### 5. **Database Health**
+- SQLite database size monitoring
+- Table and index counts
+- Integration with MemorySystem stats
+
+### 6. **Component-Specific Monitoring**
+- **Memory System**: Store/retrieve tests, cache hit rates
+- **MCP Servers**: Status of all MCP server processes
+- **ServiceNow**: API connectivity and response times
+- **Queen System**: Active sessions and agent counts
+- **Performance**: Operation success rates, response times
+
+## Usage
+
+### Basic Health Check
+
+```typescript
+import { SystemHealth } from './health/system-health';
+
+const systemHealth = new SystemHealth({
+  memory,
+  mcpManager,
+  config: {
+    checks: {
+      memory: true,
+      mcp: true,
+      servicenow: true,
+      queen: true
+    },
+    thresholds: {
+      responseTime: 1000,      // 1 second
+      memoryUsage: 0.9,        // 90%
+      cpuUsage: 0.8,           // 80%
+      queueSize: 100,
+      errorRate: 0.1           // 10%
+    }
+  }
+});
+
+await systemHealth.initialize();
+
+// Single health check
+const status = await systemHealth.performHealthCheck();
+console.log('System healthy:', status.healthy);
+console.log('CPU Usage:', status.metrics.systemResources.cpuUsage + '%');
+console.log('Memory:', status.metrics.systemResources.memoryUsage + 'MB');
+console.log('Disk:', status.metrics.systemResources.diskUsage + '%');
+```
+
+### Continuous Monitoring
+
+```typescript
+// Start monitoring every 30 seconds
+await systemHealth.startMonitoring(30000);
+
+// Listen for health events
+systemHealth.on('health:check', (status) => {
+  console.log('Health update:', status);
+});
+
+systemHealth.on('health:alert', (alert) => {
+  console.log('ALERT:', alert.message);
+});
+
+// Stop monitoring
+await systemHealth.stopMonitoring();
+```
+
+### Testing
+
+Run the test script to see real metrics:
+
+```bash
+npm run build
+npm run test:health
+```
+
+## Thresholds and Alerts
+
+The system triggers alerts when:
+- **CPU Usage** > 80% (degraded), > 90% (critical)
+- **Memory Usage** > 80% (degraded), > 90% (critical)
+- **Disk Usage** > 80% (degraded), > 90% (critical)
+- **Heap Usage** > 90% (degraded)
+- **Network Connectivity** lost (degraded)
+- **Response Times** exceed configured thresholds
+
+## Architecture
+
+```
+SystemHealth
+├── CPU Monitor (real-time calculation)
+├── Memory Monitor (system + process)
+├── Disk Monitor (platform-specific)
+├── Network Monitor (DNS-based)
+├── Component Monitors
+│   ├── Memory System
+│   ├── MCP Servers
+│   ├── ServiceNow API
+│   ├── Queen System
+│   └── Performance Metrics
+└── Alert System
+```
+
+## Platform Support
+
+- ✅ **macOS**: Full support for all metrics
+- ✅ **Linux**: Full support for all metrics
+- ✅ **Windows**: Full support (uses WMIC for disk stats)
+- ⚠️ **Other platforms**: Basic support (some metrics may be unavailable)
+
+## Performance Impact
+
+The health monitoring system is designed to have minimal performance impact:
+- CPU usage calculation adds < 1% overhead
+- Disk usage checks are cached and throttled
+- Database queries are optimized with indexes
+- Network checks are asynchronous and non-blocking
+
+## Future Enhancements
+
+- Historical metric graphing
+- Predictive failure detection
+- Custom metric plugins
+- Prometheus/Grafana integration
+- Alert webhooks and notifications

package/src/memory/README.md

@@ -0,0 +1,240 @@
+# Snow-Flow Memory System
+
+The Snow-Flow memory system provides SQLite-based persistent storage for agent coordination, artifact tracking, and performance monitoring. It's designed to meet the requirements specified in MCP_ARCHITECTURE.md with fast queries (<100ms), thread-safe operations, and automatic schema migration.
+
+## Architecture
+
+The memory system consists of three main components:
+
+### 1. SwarmMemory (`swarm-memory.ts`)
+- Core SQLite database management
+- Schema creation and migration
+- Performance optimization
+- Thread-safe operations with WAL mode
+
+### 2. MemoryOperations (`memory-operations.ts`)
+- High-level operations for data storage/retrieval
+- Agent coordination management
+- Artifact tracking
+- Performance monitoring
+- Message passing between agents
+
+### 3. MemoryClient (`memory-client.ts`)
+- Simple API for agents
+- Automatic session tracking
+- Performance tracking helpers
+- Simplified context storage
+
+## Database Schema
+
+The system implements all tables specified in MCP_ARCHITECTURE.md:
+
+- **agent_coordination**: Track agent status and progress
+- **servicenow_artifacts**: Store created artifacts
+- **agent_messages**: Inter-agent communication
+- **shared_context**: Session-wide shared data
+- **deployment_history**: Track deployments
+- **agent_dependencies**: Manage agent dependencies
+- **performance_metrics**: Performance monitoring
+
+## Usage
+
+### Basic Agent Usage
+
+```typescript
+import { MemoryClient } from './memory/index.js';
+
+// Create a memory client for an agent
+const memory = MemoryClient.forAgent(
+  'widget_creator_001',  // agent ID
+  'widget-creator',      // agent type
+  'session_123'          // session ID
+);
+
+// Register the agent
+await memory.register(['create_dashboard_widget']);
+
+// Update progress
+await memory.updateProgress(50, 'creating_template');
+
+// Store shared data
+await memory.store({
+  key: 'widget_config',
+  value: {
+    name: 'incident_dashboard',
+    features: ['charts', 'filters']
+  }
+});
+
+// Retrieve shared data
+const config = await memory.retrieve({ key: 'widget_config' });
+
+// Track performance
+const tracker = memory.trackOperation('widget_creation');
+// ... do work ...
+await tracker.complete(true);
+
+// Mark as completed
+await memory.complete();
+```
+
+### Artifact Management
+
+```typescript
+// Store artifact information
+await memory.storeArtifact({
+  sys_id: 'widget_12345',
+  type: 'widget',
+  name: 'Incident Dashboard',
+  status: 'created',
+  metadata: { version: '1.0' }
+});
+
+// Update artifact status
+await memory.updateArtifactStatus('widget_12345', 'deployed');
+
+// Find artifacts
+const widgets = await memory.findSessionArtifacts('widget');
+```
+
+### Agent Communication
+
+```typescript
+// Send handoff to another agent
+await memory.handoff({
+  to_agent: 'ui_specialist_001',
+  artifact_reference: 'widget_12345',
+  data: {
+    template_ready: true,
+    next_steps: ['add_styling', 'test_responsiveness']
+  }
+});
+
+// Check for incoming handoffs
+const handoffs = await memory.checkHandoffs();
+for (const handoff of handoffs) {
+  console.log(`Received from ${handoff.from}:`, handoff.data);
+}
+
+// Report dependency ready
+await memory.reportDependencyReady('test_agent_001', 'widget_12345');
+```
+
+### Performance Monitoring
+
+```typescript
+// Track operation performance
+const tracker = memory.trackOperation('api_call');
+try {
+  // ... perform operation ...
+  await tracker.complete(true, undefined, { 
+    response_time: 150,
+    records_processed: 100 
+  });
+} catch (error) {
+  await tracker.complete(false, error.message);
+}
+
+// Get performance statistics
+const stats = await memory.getPerformanceStats('api_call');
+console.log(`Average duration: ${stats.avgDuration}ms`);
+console.log(`Success rate: ${stats.successRate * 100}%`);
+```
+
+### Session Management
+
+```typescript
+// Get complete session state
+const state = await memory.getSessionState();
+console.log('Active agents:', state.agents);
+console.log('Created artifacts:', state.artifacts);
+console.log('Performance summary:', state.performanceSummary);
+
+// Clean up session when done
+await memory.cleanup();
+```
+
+## Advanced Usage
+
+### Direct Database Access
+
+For custom queries, you can access the underlying database:
+
+```typescript
+const db = memory.getDatabase();
+const results = db.all(`
+  SELECT * FROM servicenow_artifacts 
+  WHERE deployment_status = 'failed'
+  ORDER BY updated_at DESC
+`);
+```
+
+### Custom Operations
+
+Access the operations layer for advanced functionality:
+
+```typescript
+const ops = memory.getOperations();
+
+// Search messages with custom criteria
+const messages = await ops.searchMessages({
+  message_type: 'error',
+  processed: false,
+  limit: 10
+});
+
+// Get deployment history
+const history = await ops.getDeploymentHistory('artifact_123');
+```
+
+## Performance Optimization
+
+The memory system is optimized for performance:
+
+1. **WAL Mode**: Enables concurrent reads
+2. **Indexes**: Fast queries on common fields
+3. **Prepared Statements**: Reusable query plans
+4. **Connection Pooling**: Single shared instance
+5. **Batch Operations**: Transaction support
+
+## Best Practices
+
+1. **Always register agents** before performing operations
+2. **Use transactions** for multiple related operations
+3. **Clean up sessions** when work is complete
+4. **Track performance** for critical operations
+5. **Store minimal data** in shared context
+6. **Use appropriate expiration** for temporary data
+7. **Handle errors gracefully** with try-catch blocks
+
+## Testing
+
+Run the test suite to verify functionality:
+
+```bash
+npm run test:memory
+# or
+node dist/memory/memory-test.js
+```
+
+## Monitoring
+
+Check memory statistics:
+
+```typescript
+const stats = memory.getStats();
+console.log('Database statistics:', stats);
+
+// Run cleanup for old data
+await memory.runCleanup();
+```
+
+## Shutdown
+
+Always shutdown the memory system when done:
+
+```typescript
+MemoryClient.shutdown();
+```
+
+This ensures all connections are properly closed and data is flushed to disk.