@probelabs/visor 0.1.124 → 0.1.126

package/dist/docs/advanced-ai.md

@@ -1,4 +1,4 @@
-## 🧠 Advanced AI Features
+## Advanced AI Features
 
 ### AI Session Reuse
 Use `reuse_ai_session` on checks to continue conversation context with the AI across steps. This improves follow‑ups and consistency for follow‑on analysis and chat‑style flows.
@@ -26,6 +26,7 @@ steps:
     depends_on: [security-remediation]
     reuse_ai_session: true
     session_mode: append  # Share history - sees full conversation
+```
 
 #### Reusing your own session: `reuse_ai_session: self`
 
@@ -84,8 +85,7 @@ The corresponding testable example lives at:
 
 - `examples/session-reuse-self.yaml`
 
-This keeps the configuration small but shows how to wire `reuse_ai_session: self` and `session_mode: append` without touching higher‑level workflows like `tyk-assistant`.
-```
+This keeps the configuration small but shows how to wire `reuse_ai_session: self` and `session_mode: append` without touching higher-level workflows like `tyk-assistant`.
 
 **When to use each mode:**
 - Use **`clone`** (default) when you want parallel follow-ups that don't interfere with each other
@@ -99,61 +99,67 @@ For PR events, Visor provides comprehensive code review context:
 
 ```xml
 <pull_request>
+  <!-- Core pull request metadata including identification, branches, and change statistics -->
   <metadata>
-    <number>123</number>                    <!-- PR number -->
-    <title>Add user authentication</title>  <!-- PR title -->
-    <author>developer</author>               <!-- PR author username -->
-    <base_branch>main</base_branch>         <!-- Target branch (where changes will be merged) -->
-    <target_branch>feature-auth</target_branch> <!-- Source branch (contains the changes) -->
-    <total_additions>250</total_additions>  <!-- Total lines added across all files -->
-    <total_deletions>50</total_deletions>   <!-- Total lines removed across all files -->
-    <files_changed_count>3</files_changed_count> <!-- Number of files modified -->
+    <number>123</number>
+    <title>Add user authentication</title>
+    <author>developer</author>
+    <base_branch>main</base_branch>
+    <target_branch>feature-auth</target_branch>
+    <total_additions>250</total_additions>
+    <total_deletions>50</total_deletions>
+    <files_changed_count>3</files_changed_count>
   </metadata>
 
+  <!-- Raw diff header snippet for compatibility -->
+  <raw_diff_header>
+diff --git a/src/auth.ts b/src/auth.ts
+  </raw_diff_header>
+
+  <!-- Full pull request description provided by the author -->
   <description>
-    <!-- PR description/body text provided by the author -->
-    This PR implements JWT-based authentication with refresh token support
+This PR implements JWT-based authentication with refresh token support
   </description>
 
+  <!-- Complete unified diff showing all changes (processed with outline-diff) -->
   <full_diff>
-    <!-- Complete unified diff of all changes (present for all PR analyses) -->
-    --- src/auth.ts
-    +++ src/auth.ts
-    @@ -1,3 +1,10 @@
-    +import jwt from 'jsonwebtoken';
-    ...
+--- src/auth.ts
++++ src/auth.ts
+@@ -1,3 +1,10 @@
++import jwt from 'jsonwebtoken';
+...
   </full_diff>
 
+  <!-- Diff of only the latest commit for incremental analysis (only present for pr_updated events) -->
   <commit_diff>
-    <!-- Only present for incremental analysis (pr_updated events) -->
-    <!-- Contains diff of just the latest commit pushed -->
+<!-- Contains diff of just the latest commit pushed -->
   </commit_diff>
 
+  <!-- Summary of all files changed with statistics -->
   <files_summary>
-    <!-- List of all modified files with change statistics -->
-    <file index="1">
+    <file>
       <filename>src/auth.ts</filename>
-      <status>modified</status>          <!-- added/modified/removed/renamed -->
-      <additions>120</additions>          <!-- Lines added in this file -->
-      <deletions>10</deletions>           <!-- Lines removed from this file -->
+      <status>modified</status>
+      <additions>120</additions>
+      <deletions>10</deletions>
     </file>
   </files_summary>
 
-  <!-- Only present for issue_comment events on PRs -->
+  <!-- The comment that triggered this analysis (only present for issue_comment events) -->
   <triggering_comment>
     <author>reviewer1</author>
     <created_at>2024-01-16T15:30:00Z</created_at>
     <body>/review --check security</body>
   </triggering_comment>
 
-  <!-- Historical comments on the PR (excludes triggering comment) -->
+  <!-- Previous comments in chronological order (excluding triggering comment) -->
   <comment_history>
-    <comment index="1">
+    <comment>
       <author>reviewer2</author>
       <created_at>2024-01-15T11:00:00Z</created_at>
       <body>Please add unit tests for the authentication logic</body>
     </comment>
-    <comment index="2">
+    <comment>
       <author>developer</author>
       <created_at>2024-01-15T14:30:00Z</created_at>
       <body>Tests added in latest commit</body>
@@ -167,56 +173,57 @@ For issue events, Visor provides issue-specific context for intelligent assistan
 
 ```xml
 <issue>
+  <!-- Core issue metadata including identification, status, and timeline information -->
   <metadata>
-    <number>456</number>                   <!-- Issue number -->
-    <title>Feature request: Add dark mode</title> <!-- Issue title -->
-    <author>user123</author>                <!-- Issue author username -->
-    <state>open</state>                     <!-- Issue state: open/closed -->
-    <created_at>2024-01-15T10:30:00Z</created_at> <!-- When issue was created -->
-    <updated_at>2024-01-16T14:20:00Z</updated_at> <!-- Last update timestamp -->
-    <comments_count>5</comments_count>      <!-- Total number of comments -->
+    <number>456</number>
+    <title>Feature request: Add dark mode</title>
+    <author>user123</author>
+    <state>open</state>
+    <created_at>2024-01-15T10:30:00Z</created_at>
+    <updated_at>2024-01-16T14:20:00Z</updated_at>
+    <comments_count>5</comments_count>
   </metadata>
 
+  <!-- Full issue description and body text provided by the issue author -->
   <description>
-    <!-- Issue body/description text provided by the author -->
-    I would like to request a dark mode feature for better accessibility...
+I would like to request a dark mode feature for better accessibility...
   </description>
 
+  <!-- Applied labels for issue categorization and organization -->
   <labels>
-    <!-- GitHub labels applied to categorize the issue -->
     <label>enhancement</label>
     <label>good first issue</label>
     <label>ui/ux</label>
   </labels>
 
+  <!-- Users assigned to work on this issue -->
   <assignees>
-    <!-- Users assigned to work on this issue -->
     <assignee>developer1</assignee>
     <assignee>developer2</assignee>
   </assignees>
 
+  <!-- Associated project milestone information -->
   <milestone>
-    <!-- Project milestone this issue is part of (if any) -->
     <title>v2.0 Release</title>
-    <state>open</state>                     <!-- Milestone state: open/closed -->
-    <due_on>2024-03-01T00:00:00Z</due_on>  <!-- Milestone due date -->
+    <state>open</state>
+    <due_on>2024-03-01T00:00:00Z</due_on>
   </milestone>
 
-  <!-- Only present for issue_comment events -->
+  <!-- The comment that triggered this analysis (only present for issue_comment events) -->
   <triggering_comment>
-    <author>user456</author>                <!-- User who posted the triggering comment -->
-    <created_at>2024-01-16T15:30:00Z</created_at> <!-- When comment was posted -->
-    <body>/review security --focus authentication</body> <!-- The comment text -->
+    <author>user456</author>
+    <created_at>2024-01-16T15:30:00Z</created_at>
+    <body>/review security --focus authentication</body>
   </triggering_comment>
 
-  <!-- Historical comments on the issue (excludes triggering comment) -->
+  <!-- Previous comments in chronological order (excluding triggering comment) -->
   <comment_history>
-    <comment index="1">                     <!-- Comments ordered by creation time -->
+    <comment>
       <author>developer1</author>
       <created_at>2024-01-15T11:00:00Z</created_at>
       <body>This is a great idea! I'll start working on it.</body>
     </comment>
-    <comment index="2">
+    <comment>
       <author>user123</author>
       <created_at>2024-01-15T14:30:00Z</created_at>
       <body>Thanks! Please consider accessibility standards.</body>

package/dist/docs/ai-configuration.md

@@ -7,8 +7,8 @@ Visor supports multiple AI providers. Configure one via environment variables.
 | Provider | Env Var | Example Models |
 |----------|---------|----------------|
 | Google Gemini | `GOOGLE_API_KEY` | `gemini-2.0-flash-exp`, `gemini-1.5-pro` |
-| Anthropic Claude | `ANTHROPIC_API_KEY` | `claude-3-opus`, `claude-3-sonnet` |
-| OpenAI GPT | `OPENAI_API_KEY` | `gpt-4`, `gpt-4-turbo`, `gpt-3.5-turbo` |
+| Anthropic Claude | `ANTHROPIC_API_KEY` | `claude-3-5-sonnet-latest`, `claude-3-opus-latest` |
+| OpenAI GPT | `OPENAI_API_KEY` | `gpt-4o`, `gpt-4-turbo`, `gpt-4` |
 | AWS Bedrock | AWS credentials (see below) | `anthropic.claude-sonnet-4-20250514-v1:0` (default) |
 
 ### GitHub Actions Setup
@@ -122,10 +122,12 @@ Visor exposes Probe’s prompt controls to adjust the agent’s behavior for a g
 Accepted keys
 - Under `ai:`
   - `prompt_type`: string — Probe persona/family, e.g., `engineer`, `code-review`, `architect`.
-  - `custom_prompt`: string — Baseline/system prompt prepended by the SDK.
+  - `system_prompt`: string — Baseline/system prompt prepended by the SDK (preferred).
+  - `custom_prompt`: string — Alias for `system_prompt` (deprecated, use `system_prompt` instead).
 - At the check level (aliases if you prefer not to nest):
   - `ai_prompt_type`: string
-  - `ai_custom_prompt`: string
+  - `ai_system_prompt`: string (preferred)
+  - `ai_custom_prompt`: string (deprecated alias for `ai_system_prompt`)
   - `ai_persona`: string — optional hint we prepend as a first line: `Persona: <value>`.
 
 Examples
@@ -138,7 +140,7 @@ steps:
       provider: anthropic
       model: claude-3-5-sonnet-latest
       prompt_type: engineer
-      custom_prompt: |
+      system_prompt: |
         You are a specialist in analyzing security vulnerabilities.
         Focus on injection, authn/z, crypto, and data exposure.
     schema: code-review
@@ -148,14 +150,13 @@ steps:
   quick-architect-check:
     type: ai
     ai_prompt_type: architect     # check-level alias
-    ai_custom_prompt: "Favor modular boundaries and low coupling."
+    ai_system_prompt: "Favor modular boundaries and low coupling."
     prompt: "Assess high-level design risks in the diff"
 ```
 
 Notes
 - If `prompt_type` is omitted and a `schema` is provided, Visor defaults to `code-review`.
 - `ai_persona` is a lightweight hint added as a first line; prefer `prompt_type` when integrating with Probe personas.
-```
 
 #### AWS Bedrock Specific Configuration
 
@@ -280,7 +281,7 @@ steps:
     prompt: "Fix the security vulnerabilities found in the code"
     ai:
       provider: anthropic
-      model: claude-3-opus
+      model: claude-3-opus-latest
       allowEdit: true  # Enable Edit and Create tools
 
   read-only-review:
@@ -394,7 +395,7 @@ steps:
       - Cryptographic weaknesses
     ai:
       provider: anthropic
-      model: claude-3-opus
+      model: claude-3-opus-latest
       enableDelegate: true  # Enable task delegation to subagents
 
   focused-sql-injection-check:
@@ -434,7 +435,7 @@ steps:
     prompt: "Analyze the project structure and git status"
     ai:
       provider: anthropic
-      model: claude-3-opus
+      model: claude-3-opus-latest
       allowBash: true  # Simple one-line enable
 ```
 
@@ -527,9 +528,96 @@ steps:
 
 **Security Note:** Bash command execution respects existing security boundaries and permissions. Commands run with the same privileges as the Visor process. Always review and test bash configurations before deploying to production environments.
 
+#### Retry Configuration (`retry`)
+
+Configure automatic retries for AI provider calls when transient errors occur:
+
+```yaml
+steps:
+  resilient-review:
+    type: ai
+    prompt: "Analyze code for security vulnerabilities"
+    ai:
+      provider: anthropic
+      retry:
+        maxRetries: 3           # Maximum retry attempts (0-50)
+        initialDelay: 1000      # Initial delay in ms (0-60000)
+        maxDelay: 30000         # Maximum delay cap in ms (0-300000)
+        backoffFactor: 2        # Exponential backoff multiplier (1-10)
+        retryableErrors:        # Custom error patterns to retry on
+          - "rate limit"
+          - "timeout"
+```
+
+**Configuration Options:**
+
+- **`maxRetries`** (number): Maximum retry attempts. Default varies by provider.
+- **`initialDelay`** (number): Initial delay between retries in milliseconds.
+- **`maxDelay`** (number): Maximum delay cap to prevent excessive waits.
+- **`backoffFactor`** (number): Multiplier for exponential backoff between retries.
+- **`retryableErrors`** (string[]): Custom error message patterns that should trigger retries.
+
+#### Fallback Configuration (`fallback`)
+
+Configure fallback providers when the primary AI provider fails:
+
+```yaml
+steps:
+  fault-tolerant-review:
+    type: ai
+    prompt: "Review code for quality issues"
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-latest
+      fallback:
+        strategy: custom        # 'same-model', 'same-provider', 'any', or 'custom'
+        maxTotalAttempts: 5     # Maximum attempts across all providers
+        auto: true              # Auto-detect fallbacks from available env vars
+        providers:              # Custom fallback chain
+          - provider: openai
+            model: gpt-4o
+          - provider: google
+            model: gemini-2.0-flash-exp
+```
+
+**Configuration Options:**
+
+- **`strategy`** (string): Fallback strategy:
+  - `same-model`: Retry with the same model
+  - `same-provider`: Try different models from the same provider
+  - `any`: Try any available provider
+  - `custom`: Use the specified providers list
+- **`providers`** (array): Array of fallback provider configurations
+- **`maxTotalAttempts`** (number): Maximum total attempts across all providers
+- **`auto`** (boolean): Automatically detect and use fallback providers from environment variables
+
+#### Completion Prompt (`completion_prompt`)
+
+Run a validation or review prompt after the AI completes its primary task:
+
+```yaml
+steps:
+  validated-review:
+    type: ai
+    prompt: "Analyze the codebase for security issues"
+    ai:
+      provider: anthropic
+      completion_prompt: |
+        Review your analysis above. Verify that:
+        1. All findings have specific file and line references
+        2. Severity levels are appropriate
+        3. Recommendations are actionable
+        If any issues are found, revise your response.
+```
+
+**When to use completion prompts:**
+- Validate AI output meets quality standards
+- Self-review for accuracy and completeness
+- Ensure proper formatting of responses
+
 ### Fallback Behavior
 
 If no key is configured, Visor falls back to fast, heuristic checks (simple patterns, basic style/perf). For best results, set a provider.
 
 ### MCP (Tools) Support
-See docs/mcp.md for adding MCP servers (Probe, Jira, Filesystem, etc.).
+See [mcp.md](./mcp.md) for adding MCP servers (Probe, Jira, Filesystem, etc.).

package/dist/docs/ai-custom-tools-usage.md

@@ -2,8 +2,28 @@
 
 ## TL;DR
 
-You can expose custom shell-based tools to AI by adding `tools: [tool-names]` to your `ai_mcp_servers` configuration. No new config sections needed!
+You can expose custom shell-based tools to AI using one of two methods:
 
+**Method 1: Using `ai_custom_tools` (Recommended)**
+```yaml
+tools:
+  grep-pattern:
+    name: grep-pattern
+    exec: 'grep -rn "{{ args.pattern }}" *.ts'
+    inputSchema:
+      type: object
+      properties:
+        pattern: {type: string}
+      required: [pattern]
+
+steps:
+  security-scan:
+    type: ai
+    prompt: Use grep-pattern to find security issues.
+    ai_custom_tools: [grep-pattern]  # Simple and explicit
+```
+
+**Method 2: Using `tools:` within `ai_mcp_servers`**
 ```yaml
 tools:
   grep-pattern:
@@ -21,14 +41,14 @@ steps:
     prompt: Use grep-pattern to find security issues.
     ai_mcp_servers:
       custom-tools:
-        tools: [grep-pattern]  # ← Automatically creates ephemeral SSE MCP server!
+        tools: [grep-pattern]  # Creates ephemeral SSE MCP server
 ```
 
 ## How It Works
 
-When you add `tools: [...]` to an MCP server config:
+When you use either `ai_custom_tools` or `tools: [...]` within an MCP server config:
 
-1. **Automatic Detection**: Visor detects you're referencing custom tools (not a command/URL)
+1. **Automatic Detection**: Visor detects you're referencing custom tools defined in the global `tools:` section
 2. **Server Startup**: Creates an ephemeral SSE MCP server on an available port
 3. **Tool Exposure**: Your custom tools become available to the AI via MCP protocol
 4. **Auto Cleanup**: Server stops automatically when the AI check completes
@@ -51,9 +71,7 @@ steps:
   security-review:
     type: ai
     prompt: Use check-secrets to find hardcoded credentials.
-    ai_mcp_servers:
-      security-tools:
-        tools: [check-secrets]
+    ai_custom_tools: [check-secrets]
 ```
 
 ### Example 2: Multiple Custom Tools
@@ -82,9 +100,7 @@ steps:
     prompt: |
       Use grep-pattern to find console.log statements.
       Use count-todos to count pending work items.
-    ai_mcp_servers:
-      custom-tools:
-        tools: [grep-pattern, count-todos]
+    ai_custom_tools: [grep-pattern, count-todos]
 ```
 
 ### Example 3: Combining Custom Tools with External MCP Servers
@@ -96,11 +112,10 @@ steps:
     prompt: |
       You have both custom tools and external MCP servers.
       Use them for a comprehensive review.
+    # Custom tools via ai_custom_tools
+    ai_custom_tools: [grep-pattern, check-secrets]
+    # External MCP servers
     ai_mcp_servers:
-      # Custom tools via ephemeral SSE server
-      my-tools:
-        tools: [grep-pattern, check-secrets]
-      # External MCP server via stdio
       filesystem:
         command: npx
         args: ["-y", "@modelcontextprotocol/server-filesystem", "."]
@@ -108,7 +123,22 @@ steps:
 
 ## Configuration Format
 
-### Custom Tools Server (Ephemeral SSE)
+### Method 1: ai_custom_tools (Recommended)
+
+```yaml
+steps:
+  my-check:
+    type: ai
+    ai_custom_tools: [tool-name-1, tool-name-2, ...]
+```
+
+**Key points:**
+- Simple and explicit configuration
+- Tools must be defined in global `tools:` section
+- Automatically creates SSE server on ephemeral port
+- Can be combined with `ai_mcp_servers` for external servers
+
+### Method 2: tools: within ai_mcp_servers
 
 ```yaml
 ai_mcp_servers:
@@ -117,12 +147,12 @@ ai_mcp_servers:
 ```
 
 **Key points:**
-- Server name can be anything (e.g., `custom-tools`, `my-tools`, `security-tools`)
+- Server name can be anything (e.g., `custom-tools`, `my-tools`)
 - Tools must be defined in global `tools:` section
 - Automatically creates SSE server on ephemeral port
 - No command/URL needed - handled automatically
 
-### External MCP Server (Traditional)
+### External MCP Server (for comparison)
 
 ```yaml
 ai_mcp_servers:
@@ -136,22 +166,32 @@ ai_mcp_servers:
 - Uses stdio transport by default
 - Can also use SSE/HTTP with `url:` and `transport:`
 
-## Backward Compatibility
+## Both Methods Work
 
-The old `ai_custom_tools` field still works for backward compatibility:
+Both configuration methods are fully supported:
 
 ```yaml
 steps:
-  my-check:
+  # Method 1: ai_custom_tools (recommended for simplicity)
+  check-with-custom-tools:
+    type: ai
+    ai_custom_tools: [tool1, tool2]
+
+  # Method 2: tools: in ai_mcp_servers (useful for naming)
+  check-with-mcp-tools:
     type: ai
-    ai_custom_tools: [tool1, tool2]  # Still works!
+    ai_mcp_servers:
+      my-security-tools:
+        tools: [tool1, tool2]
 ```
 
-But the **recommended approach** is to use `ai_mcp_servers` with `tools:` because:
-- ✅ Consistent with existing MCP server configuration
-- ✅ No new config fields to learn
-- ✅ Easier to combine custom + external MCP servers
-- ✅ More flexible (can name your tool server)
+**Choose `ai_custom_tools` when:**
+- You want simple, explicit configuration
+- You're combining custom tools with external MCP servers
+
+**Choose `tools:` in `ai_mcp_servers` when:**
+- You want to name your tool server
+- You prefer all MCP configuration in one place
 
 ## Complete Example
 
@@ -200,9 +240,7 @@ steps:
       1. Use grep-security to find eval(), exec(), or dangerous patterns
       2. Use scan-dependencies to check for outdated packages
       3. Report findings with severity levels
-    ai_mcp_servers:
-      security-tools:
-        tools: [grep-security, scan-dependencies]
+    ai_custom_tools: [grep-security, scan-dependencies]
     ai:
       provider: anthropic
       model: claude-3-5-sonnet-20241022
@@ -213,9 +251,7 @@ steps:
       Analyze code metrics:
       - Use count-lines to measure TypeScript code
       - Provide insights on code size and complexity
-    ai_mcp_servers:
-      metrics-tools:
-        tools: [count-lines]
+    ai_custom_tools: [count-lines]
     ai:
       provider: anthropic
       model: claude-3-5-sonnet-20241022
@@ -250,7 +286,8 @@ output:
 **Problem**: AI says it doesn't have access to tools
 
 **Solutions**:
-- Verify `tools:` field in `ai_mcp_servers` (not `tool` singular)
+- Verify `ai_custom_tools:` syntax (not `ai_custom_tool` singular)
+- If using `ai_mcp_servers`, verify `tools:` field (not `tool` singular)
 - Check tool names match exactly (case-sensitive)
 - Enable debug logging to see server startup