@probelabs/visor 0.1.107 → 0.1.112

package/dist/docs/ai-custom-tools-usage.md

@@ -0,0 +1,261 @@
+# Using Custom Tools with AI (Simple Guide)
+
+## TL;DR
+
+You can expose custom shell-based tools to AI by adding `tools: [tool-names]` to your `ai_mcp_servers` configuration. No new config sections needed!
+
+```yaml
+tools:
+  grep-pattern:
+    name: grep-pattern
+    exec: 'grep -rn "{{ args.pattern }}" *.ts'
+    inputSchema:
+      type: object
+      properties:
+        pattern: {type: string}
+      required: [pattern]
+
+steps:
+  security-scan:
+    type: ai
+    prompt: Use grep-pattern to find security issues.
+    ai_mcp_servers:
+      custom-tools:
+        tools: [grep-pattern]  # ← Automatically creates ephemeral SSE MCP server!
+```
+
+## How It Works
+
+When you add `tools: [...]` to an MCP server config:
+
+1. **Automatic Detection**: Visor detects you're referencing custom tools (not a command/URL)
+2. **Server Startup**: Creates an ephemeral SSE MCP server on an available port
+3. **Tool Exposure**: Your custom tools become available to the AI via MCP protocol
+4. **Auto Cleanup**: Server stops automatically when the AI check completes
+
+## Examples
+
+### Example 1: Security Scanning
+
+```yaml
+tools:
+  check-secrets:
+    name: check-secrets
+    description: Scan for hardcoded secrets
+    exec: 'grep -rn -E "(api_key|secret|password)" .'
+    inputSchema:
+      type: object
+      properties: {}
+
+steps:
+  security-review:
+    type: ai
+    prompt: Use check-secrets to find hardcoded credentials.
+    ai_mcp_servers:
+      security-tools:
+        tools: [check-secrets]
+```
+
+### Example 2: Multiple Custom Tools
+
+```yaml
+tools:
+  grep-pattern:
+    name: grep-pattern
+    exec: 'grep -rn "{{ args.pattern }}" *.ts'
+    inputSchema:
+      type: object
+      properties:
+        pattern: {type: string}
+      required: [pattern]
+
+  count-todos:
+    name: count-todos
+    exec: 'grep -r "TODO" src/ | wc -l'
+    inputSchema:
+      type: object
+      properties: {}
+
+steps:
+  code-quality:
+    type: ai
+    prompt: |
+      Use grep-pattern to find console.log statements.
+      Use count-todos to count pending work items.
+    ai_mcp_servers:
+      custom-tools:
+        tools: [grep-pattern, count-todos]
+```
+
+### Example 3: Combining Custom Tools with External MCP Servers
+
+```yaml
+steps:
+  full-review:
+    type: ai
+    prompt: |
+      You have both custom tools and external MCP servers.
+      Use them for a comprehensive review.
+    ai_mcp_servers:
+      # Custom tools via ephemeral SSE server
+      my-tools:
+        tools: [grep-pattern, check-secrets]
+      # External MCP server via stdio
+      filesystem:
+        command: npx
+        args: ["-y", "@modelcontextprotocol/server-filesystem", "."]
+```
+
+## Configuration Format
+
+### Custom Tools Server (Ephemeral SSE)
+
+```yaml
+ai_mcp_servers:
+  <server-name>:
+    tools: [tool-name-1, tool-name-2, ...]
+```
+
+**Key points:**
+- Server name can be anything (e.g., `custom-tools`, `my-tools`, `security-tools`)
+- Tools must be defined in global `tools:` section
+- Automatically creates SSE server on ephemeral port
+- No command/URL needed - handled automatically
+
+### External MCP Server (Traditional)
+
+```yaml
+ai_mcp_servers:
+  <server-name>:
+    command: <command>
+    args: [<arg1>, <arg2>, ...]
+```
+
+**Key points:**
+- Requires command to run external MCP server
+- Uses stdio transport by default
+- Can also use SSE/HTTP with `url:` and `transport:`
+
+## Backward Compatibility
+
+The old `ai_custom_tools` field still works for backward compatibility:
+
+```yaml
+steps:
+  my-check:
+    type: ai
+    ai_custom_tools: [tool1, tool2]  # Still works!
+```
+
+But the **recommended approach** is to use `ai_mcp_servers` with `tools:` because:
+- ✅ Consistent with existing MCP server configuration
+- ✅ No new config fields to learn
+- ✅ Easier to combine custom + external MCP servers
+- ✅ More flexible (can name your tool server)
+
+## Complete Example
+
+```yaml
+version: "1.0"
+
+# Define custom tools (global section)
+tools:
+  grep-security:
+    name: grep-security
+    description: Search for security-related patterns
+    inputSchema:
+      type: object
+      properties:
+        pattern: {type: string}
+      required: [pattern]
+    exec: 'grep -rn "{{ args.pattern }}" src/'
+    parseJson: false
+
+  scan-dependencies:
+    name: scan-dependencies
+    description: Check for outdated dependencies
+    inputSchema:
+      type: object
+      properties: {}
+    exec: 'npm outdated --json 2>/dev/null || echo "{}"'
+    parseJson: true
+
+  count-lines:
+    name: count-lines
+    description: Count lines of code
+    inputSchema:
+      type: object
+      properties:
+        extension: {type: string}
+      required: [extension]
+    exec: 'find . -name "*.{{ args.extension }}" -exec wc -l {} + | tail -1'
+    parseJson: false
+
+# Use custom tools in AI checks
+steps:
+  security-audit:
+    type: ai
+    prompt: |
+      Perform a security audit:
+      1. Use grep-security to find eval(), exec(), or dangerous patterns
+      2. Use scan-dependencies to check for outdated packages
+      3. Report findings with severity levels
+    ai_mcp_servers:
+      security-tools:
+        tools: [grep-security, scan-dependencies]
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+
+  code-metrics:
+    type: ai
+    prompt: |
+      Analyze code metrics:
+      - Use count-lines to measure TypeScript code
+      - Provide insights on code size and complexity
+    ai_mcp_servers:
+      metrics-tools:
+        tools: [count-lines]
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+
+output:
+  pr_comment:
+    enabled: true
+    format: markdown
+    group_by: check
+    collapse: false
+```
+
+## Troubleshooting
+
+### "Custom tool not found"
+
+**Problem**: `Custom tool not found: my-tool`
+
+**Solution**: Make sure the tool is defined in the global `tools:` section with the exact same name.
+
+### "Failed to start custom tools SSE server"
+
+**Problem**: Server fails to start
+
+**Solutions**:
+- Check if ports are available
+- Verify tool definitions are valid
+- Enable debug mode: `ai.debug: true`
+
+### Tools not appearing to AI
+
+**Problem**: AI says it doesn't have access to tools
+
+**Solutions**:
+- Verify `tools:` field in `ai_mcp_servers` (not `tool` singular)
+- Check tool names match exactly (case-sensitive)
+- Enable debug logging to see server startup
+
+## See Also
+
+- Full documentation: `docs/ai-custom-tools.md`
+- Examples: `examples/ai-custom-tools-simple.yaml`
+- Advanced examples: `examples/ai-custom-tools-example.yaml`

package/dist/docs/ai-custom-tools.md

@@ -0,0 +1,392 @@
+# AI Custom Tools via Ephemeral SSE MCP Servers
+
+## Overview
+
+This feature allows AI checks to use custom shell-based tools defined in your Visor configuration. Custom tools are automatically exposed to AI via ephemeral SSE (Server-Sent Events) MCP (Model Context Protocol) servers that start on-demand and clean up automatically.
+
+## Key Benefits
+
+✅ **Zero Configuration**: Ports are automatically assigned by the OS
+✅ **Automatic Lifecycle**: Servers start before AI execution and clean up after
+✅ **Seamless Integration**: Works with existing AI providers (Anthropic, OpenAI, Gemini)
+✅ **Secure Execution**: Reuses existing tool security features
+✅ **Concurrent Support**: Multiple AI checks run independently with separate servers
+
+## How It Works
+
+1. **Define Tools**: Create custom tools in the `tools:` section of your config
+2. **Reference in AI Check**: Add `ai_custom_tools:` to your AI check configuration
+3. **Automatic Server**: Visor starts an ephemeral SSE MCP server on an available port
+4. **AI Execution**: AI can call your custom tools via the MCP protocol
+5. **Automatic Cleanup**: Server stops automatically when the AI check completes
+
+```
+┌─────────────┐
+│   AI Check  │
+│  (Step 1)   │
+└──────┬──────┘
+       │
+       │ ai_custom_tools: [grep, scan]
+       ↓
+┌─────────────────────────────┐
+│  AICheckProvider            │
+│  1. Detect custom tools     │
+│  2. Start SSE server (port) │
+│  3. Add to MCP servers      │
+└──────────┬──────────────────┘
+           │
+           ↓
+    ┌──────────────────┐
+    │  SSE MCP Server  │
+    │  localhost:PORT  │
+    │                  │
+    │  Tools:          │
+    │  - grep-tool     │
+    │  - scan-tool     │
+    └──────────────────┘
+           ↑
+           │ MCP Protocol (tools/list, tools/call)
+           │
+    ┌──────────────────┐
+    │   AI Provider    │
+    │  (Claude, GPT)   │
+    └──────────────────┘
+```
+
+## Configuration
+
+### Basic Example
+
+```yaml
+version: "1.0"
+
+# Define custom tools
+tools:
+  grep-pattern:
+    name: grep-pattern
+    description: Search for patterns in files
+    inputSchema:
+      type: object
+      properties:
+        pattern:
+          type: string
+          description: The regex pattern to search
+      required: [pattern]
+    exec: 'grep -rn "{{ args.pattern }}" *.ts'
+    parseJson: false
+
+steps:
+  security-review:
+    type: ai
+    prompt: |
+      Use the grep-pattern tool to find potential security issues.
+      Search for: eval, exec, dangerouslySetInnerHTML
+    ai_custom_tools:
+      - grep-pattern  # ← Enable custom tool for this AI check
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+```
+
+### Advanced Example with Multiple Tools
+
+```yaml
+tools:
+  check-secrets:
+    name: check-secrets
+    description: Scan for hardcoded secrets
+    inputSchema:
+      type: object
+      properties:
+        file:
+          type: string
+          description: File to scan (optional)
+    exec: |
+      grep -rn -E "(api[_-]?key|secret|password)" {{ args.file | default: "." }}
+    parseJson: false
+    timeout: 10000
+
+  count-todos:
+    name: count-todos
+    description: Count TODO comments
+    inputSchema:
+      type: object
+      properties: {}
+    exec: 'grep -r "TODO" src/ | wc -l'
+    parseJson: false
+
+  file-stats:
+    name: file-stats
+    description: Get file statistics
+    inputSchema:
+      type: object
+      properties:
+        filename:
+          type: string
+          description: File to analyze
+      required: [filename]
+    exec: |
+      echo "Lines: $(wc -l < {{ args.filename }})"
+      echo "Size: $(wc -c < {{ args.filename }}) bytes"
+    parseJson: false
+
+steps:
+  comprehensive-review:
+    type: ai
+    prompt: |
+      You have access to specialized analysis tools:
+      - check-secrets: Scan for hardcoded credentials
+      - count-todos: Count pending work items
+      - file-stats: Analyze file statistics
+
+      Use these tools to provide a comprehensive code review.
+    ai_custom_tools:
+      - check-secrets
+      - count-todos
+      - file-stats
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      debug: true
+```
+
+### Combining with External MCP Servers
+
+You can combine custom tools with external MCP servers:
+
+```yaml
+steps:
+  full-review:
+    type: ai
+    prompt: |
+      You have both custom tools and external MCP servers available.
+      Use them strategically for a thorough review.
+    ai_custom_tools:
+      - grep-pattern
+      - check-secrets
+    ai_mcp_servers:
+      filesystem:
+        command: npx
+        args: ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
+      probe:
+        command: npx
+        args: ["-y", "@probelabs/probe@latest", "mcp"]
+```
+
+## Tool Definition Reference
+
+### Required Fields
+
+- `name`: Unique identifier for the tool
+- `exec`: Shell command to execute (supports Liquid templates)
+- `inputSchema`: JSON Schema defining the tool's input parameters
+
+### Optional Fields
+
+- `description`: Human-readable description of what the tool does
+- `parseJson`: Whether to parse the command output as JSON (default: false)
+- `timeout`: Execution timeout in milliseconds (default: 30000)
+- `stdin`: Optional input to pass to the command
+- `transform`: Liquid template to transform the output
+- `transform_js`: JavaScript expression to transform the output
+- `cwd`: Working directory for command execution
+- `env`: Environment variables to set
+
+### Input Schema
+
+The `inputSchema` uses JSON Schema format to define tool parameters:
+
+```yaml
+inputSchema:
+  type: object
+  properties:
+    pattern:
+      type: string
+      description: Pattern to search for
+    files:
+      type: string
+      description: File glob pattern
+      default: "*.ts"
+  required: [pattern]  # List of required parameters
+```
+
+### Liquid Templates
+
+You can use Liquid templates in `exec`, `stdin`, and `transform`:
+
+```yaml
+exec: 'grep -n "{{ args.pattern }}" {{ args.files | default: "*.ts" }}'
+```
+
+Available variables:
+- `args`: Tool input arguments
+- `pr`: PR information (if available)
+- `files`: Changed files (if available)
+- `outputs`: Outputs from previous checks
+- `env`: Environment variables
+
+## Security Considerations
+
+Custom tools run with the same security constraints as other command providers:
+
+1. **Command Injection Protection**: Input validation via JSON Schema
+2. **Sandboxed Execution**: Tools run in isolated processes
+3. **Timeout Enforcement**: All tools have configurable timeouts
+4. **Localhost Only**: SSE servers bind only to localhost
+5. **No Credential Storage**: Servers don't store authentication data
+
+## Troubleshooting
+
+### Server Won't Start
+
+**Problem**: `Failed to start custom tools SSE server`
+
+**Solutions**:
+- Check if ports are available (firewall/permissions)
+- Verify tools are defined in global `tools:` section
+- Enable debug mode: `ai.debug: true`
+
+### Tool Not Found
+
+**Problem**: `Custom tool not found: <tool-name>`
+
+**Solutions**:
+- Verify tool name matches exactly (case-sensitive)
+- Check tool is defined in global `tools:` section
+- Ensure `ai_custom_tools` references correct tool name
+
+### Tool Execution Timeout
+
+**Problem**: Tool calls timeout
+
+**Solutions**:
+- Increase tool timeout: `timeout: 60000` (60 seconds)
+- Simplify the tool command
+- Check command is not blocking on input
+
+### Debug Mode
+
+Enable debug logging to see detailed server operations:
+
+```yaml
+steps:
+  my-check:
+    type: ai
+    ai_custom_tools: [my-tool]
+    ai:
+      debug: true  # ← Enable debug logging
+```
+
+You'll see:
+- Server startup messages
+- Port assignment
+- Tool execution logs
+- Cleanup operations
+
+## Performance
+
+- **Server Startup**: < 100ms
+- **Tool Execution**: Inherits timeout from tool config (default: 30s)
+- **Server Shutdown**: < 1s graceful, 5s forced
+- **Memory Overhead**: Minimal per server instance
+- **Concurrent Requests**: Queued (one at a time per server)
+
+## Implementation Details
+
+### Components
+
+1. **CustomToolsSSEServer** (`src/providers/mcp-custom-sse-server.ts`)
+   - HTTP server with SSE endpoint
+   - MCP protocol implementation
+   - Tool execution via CustomToolExecutor
+
+2. **AICheckProvider Integration** (`src/providers/ai-check-provider.ts`)
+   - Automatic tool detection
+   - Server lifecycle management
+   - MCP server configuration injection
+
+3. **Configuration Types** (`src/types/config.ts`)
+   - `ai_custom_tools?: string[]` field on CheckConfig
+
+### MCP Protocol Support
+
+The server implements these MCP methods:
+
+- `initialize`: Connection initialization
+- `tools/list`: List available tools
+- `tools/call`: Execute a tool
+- `notifications/initialized`: Initialization confirmation
+
+Message format: JSON-RPC 2.0
+
+### Lifecycle Management
+
+```typescript
+// Pseudo-code of lifecycle
+const server = new CustomToolsSSEServer(tools, sessionId, debug);
+
+try {
+  const port = await server.start();  // OS assigns port
+  // ... AI execution with tools ...
+} finally {
+  await server.stop();  // Always cleanup
+}
+```
+
+## Examples
+
+See `examples/ai-custom-tools-example.yaml` for a comprehensive example with:
+- Security scanning tools
+- Code quality tools
+- File analysis tools
+- Git integration tools
+- Combined custom + external MCP servers
+
+## Testing
+
+Run the manual test to verify functionality:
+
+```bash
+npx ts-node manual-test-simple.ts
+```
+
+Expected output:
+```
+============================================================
+🧪 CustomToolsSSEServer - Quick Manual Test
+============================================================
+
+1️⃣  Starting SSE server...
+   ✅ Server running on http://localhost:58869/sse
+
+2️⃣  Calling echo-tool with message...
+   ✅ Response: ✅ Tool works: Hello from custom tools!
+
+3️⃣  Calling pwd-tool...
+   ✅ Working directory: /path/to/project
+
+4️⃣  Testing error handling (invalid tool)...
+   ✅ Error correctly returned: Internal error
+
+============================================================
+✅ ALL TESTS PASSED! Feature is working correctly! 🎉
+============================================================
+```
+
+## Future Enhancements
+
+Potential improvements:
+- Tool result caching
+- Parallel tool execution
+- Tool dependencies
+- Tool composition (chaining)
+- Persistent MCP servers (optional)
+- Tool metrics and monitoring
+
+## Support
+
+For issues or questions:
+- Check troubleshooting section above
+- Enable debug mode for detailed logs
+- Review test files in `tests/unit/mcp-custom-sse-server.test.ts`
+- See examples in `examples/ai-custom-tools-example.yaml`

package/dist/docs/bot-transports-rfc.md

@@ -0,0 +1,23 @@
+# RFC: Bot Transports for Visor (Slack-first)
+
+Status: Draft
+
+This RFC proposes a Slack integration built on the event-bus/state-machine engine. The first iteration focuses on:
+- A Slack frontend that subscribes to engine events and posts an evolving message per group (e.g., overview, review).
+- Simple configuration via `frontends` in the workflow config, e.g.
+
+```
+frontends:
+  - name: slack
+    config:
+      defaultChannel: C12345678
+      groupChannels:
+        overview: C87654321
+```
+
+Design notes:
+- No placeholder/queued messages are posted; only content-producing events produce/modify messages.
+- Messages are updated in-place (using `chat.update`) keyed by group. We do not rely on hidden markers in message text.
+- Debounce/coalescing reduces API churn during bursts; terminal state forces an immediate flush.
+- Future work will add inbound Slack handling (webhooks) to trigger workflows and attach conversation context.
+

package/dist/docs/configuration.md

@@ -81,6 +81,27 @@ steps:
     prompt: "Analyze code for security vulnerabilities"
 ```
 
+### Lifecycle Hooks
+
+Use `on_init` to run preprocessing tasks before a step executes:
+
+```yaml
+steps:
+  ai-review:
+    type: ai
+    on_init:
+      run:
+        - tool: fetch-jira-issue
+          with:
+            issue_key: "{{ pr.title | regex_search: '[A-Z]+-[0-9]+' }}"
+          as: jira-data
+    prompt: |
+      Review this PR considering JIRA issue context:
+      {{ outputs['jira-data'] | json }}
+```
+
+See [Lifecycle Hooks](./lifecycle-hooks.md) for complete documentation.
+
 ### Environment Variables
 
 Inject environment variables globally or per-check via `env`: