@probelabs/visor 0.1.124 → 0.1.126

package/dist/docs/testing/troubleshooting.md

@@ -35,8 +35,9 @@ Common issues and how to fix them quickly.
 
 ## How do I see more logs?
 
-- Set `VISOR_DEBUG=true`.
-- To isolate the problem, run a single stage: `visor test --only case#stage`.
+- Use the `--debug` CLI flag or set `VISOR_DEBUG=true`.
+- To isolate the problem, run a single stage: `visor test --only case#stage` (name substring match) or `--only case#N` (1-based index).
+- Example: `visor test --debug --only pr-review-e2e-flow#facts-invalid`
 
 ## Pause at “on_finish: no result found … — skip”
 
@@ -53,3 +54,39 @@ What we changed: we added an early return in `handleOnFinishHooks` when there ar
 Downsides: none functionally. The only trade‑off is that debug visibility is slightly reduced in that specific “no parents ran” case; enable `VISOR_DEBUG=true` if you need to trace the discovery step regardless.
 
 Guarantee: if a check executed and defines `on_finish`, `on_finish` still executes for that check once its forEach finishes. The early return only triggers when no eligible parent produced results in the run.
+
+## Mock structure does not match expected output
+
+- Ensure mock shapes match what the step schema expects. For AI steps with a schema, provide structured fields directly (not wrapped in `returns:`).
+- For command/HTTP provider steps, include `stdout`, `exit_code`, `status`, or `body` as appropriate.
+- Review [Fixtures and Mocks](./fixtures-and-mocks.md) for detailed mock examples.
+
+## Flow stage fails unexpectedly
+
+- Each stage computes coverage as a delta from the previous stage. If a step from a prior stage executes again, you need to account for it in the current stage's `expect.calls`.
+- Check that mocks are merged correctly: stage mocks override flow-level mocks (`{...flow.mocks, ...stage.mocks}`).
+- Use `--only case#stage` to isolate and debug a single stage.
+
+## CI test runs slower than expected
+
+- Ensure `ai_provider: mock` is set in `tests.defaults` for offline, fast execution.
+- Use `--max-parallel` to run cases concurrently within a suite.
+- Use `--max-suites` when running multiple test files.
+- Consider `--prompt-max-chars` to reduce memory usage for large diffs.
+
+## Tests pass locally but fail in CI
+
+- Check for environment variable differences. CI auto-detects and adjusts some defaults (e.g., `VISOR_TEST_PROMPT_MAX_CHARS`).
+- Ensure fixtures don't depend on local file paths or network access.
+- Run with `--debug` in CI to capture more diagnostic output.
+
+## Related Documentation
+
+- [Getting Started](./getting-started.md) - Introduction to the test framework
+- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
+- [Assertions](./assertions.md) - Available assertion types
+- [Fixtures and Mocks](./fixtures-and-mocks.md) - Managing test data
+- [Flows](./flows.md) - Multi-stage test flows
+- [Cookbook](./cookbook.md) - Copy-pasteable test recipes
+- [CLI](./cli.md) - Test runner command line options
+- [CI Integration](./ci.md) - Running tests in CI pipelines

package/dist/docs/timeouts.md

@@ -1,6 +1,6 @@
-# Timeouts: configuration and behavior
+# Timeouts: Configuration and Behavior
 
-Checks can specify a per‑check timeout. When a timeout is reached, the provider is aborted and the engine records a timeout error and skips direct dependents.
+Checks can specify a per-check timeout. When a timeout is reached, the provider is aborted and the engine records a timeout error and skips direct dependents.
 
 ## Configuration
 
@@ -8,31 +8,149 @@ Checks can specify a per‑check timeout. When a timeout is reached, the provide
 steps:
   fetch-tickets:
     type: command
-    timeout: 300  # seconds (command)
+    timeout: 60  # seconds (command provider uses seconds)
     exec: node scripts/jira/batch-fetch.js "$JQL" --limit $LIMIT
 ```
 
-Provider units and defaults:
+## Provider-Specific Timeout Behavior
 
-- `command`:
-  - Units: seconds.
-  - Default: 60s if not specified.
-  - Effect on timeout: issue `command/timeout` with message “Command execution timed out after <N> seconds”. Direct dependents are skipped.
-- `http_client`:
-  - Units: milliseconds.
-  - Default: provider default (see provider docs/test for specifics).
-- `ai`:
-  - Use `ai.timeout` (milliseconds). CLI `--timeout` also maps to AI timeout.
+Each provider has its own timeout handling:
 
-The engine propagates the per‑check timeout into providers in dependency‑aware execution so behavior is consistent even in multi‑check workflows.
+| Provider | Unit | Default | Config Key |
+|----------|------|---------|------------|
+| `command` | seconds | 60s | `timeout` |
+| `http_client` | milliseconds | 30000ms (30s) | `timeout` |
+| `mcp` | seconds | 60s | `timeout` |
+| `ai` | milliseconds | varies by model | `ai.timeout` |
+| `human-input` | seconds | none (waits indefinitely) | `timeout` |
+| `custom tools` | milliseconds | 30000ms (30s) | `timeout` |
+| `git-checkout` | milliseconds | 300000ms (5 min) | `clone_timeout_ms` |
+
+### Command Provider
+
+- **Units**: seconds
+- **Default**: 60 seconds
+- **Error rule ID**: `command/timeout`
+- **Message**: "Command execution timed out after N seconds"
+- **Behavior**: Dependents are skipped with reason `dependency_failed`
+
+```yaml
+steps:
+  long-build:
+    type: command
+    timeout: 300  # 5 minutes
+    exec: npm run build
+```
+
+### HTTP Client Provider
+
+- **Units**: milliseconds
+- **Default**: 30000ms (30 seconds)
+- **Error rule ID**: `http_client/fetch_error` or `http_client/download_timeout`
+- **Message**: "Request timed out after Nms"
+
+```yaml
+steps:
+  fetch-data:
+    type: http_client
+    timeout: 60000  # 60 seconds
+    url: https://api.example.com/data
+```
+
+### MCP Provider
+
+- **Units**: seconds (converted to milliseconds internally)
+- **Default**: 60 seconds
+- **Applies to**: Connection timeout and request timeout separately
+
+```yaml
+steps:
+  mcp-analysis:
+    type: mcp
+    timeout: 120  # 2 minutes
+    command: npx
+    args: ["-y", "@probelabs/probe@latest", "mcp"]
+    method: search_code
+```
+
+### AI Provider
+
+- **Units**: milliseconds
+- **Default**: varies by provider/model
+- **Config location**: Nested under `ai:` block
+
+```yaml
+steps:
+  ai-review:
+    type: ai
+    prompt: "Review this code"
+    ai:
+      provider: anthropic
+      model: claude-3-opus
+      timeout: 120000  # 2 minutes
+```
+
+### Human Input Provider
+
+- **Units**: seconds
+- **Default**: no timeout (waits indefinitely)
+- **Behavior**: Returns default value if timeout expires
+
+```yaml
+steps:
+  approval:
+    type: human-input
+    prompt: "Approve deployment? (yes/no)"
+    timeout: 300  # 5 minutes
+    default: "no"
+```
+
+### Custom Tools
+
+- **Units**: milliseconds
+- **Default**: 30000ms (30 seconds)
+
+```yaml
+tools:
+  my-tool:
+    name: my-tool
+    exec: ./slow-script.sh
+    timeout: 60000  # 60 seconds
+```
+
+### Git Checkout Provider
+
+- **Units**: milliseconds
+- **Default**: 300000ms (5 minutes)
+- **Config key**: `clone_timeout_ms`
+
+```yaml
+steps:
+  checkout-repo:
+    type: git-checkout
+    ref: main
+    clone_timeout_ms: 600000  # 10 minutes for large repos
+```
+
+## CLI Global Timeout
+
+The CLI supports a global timeout via the `--timeout` flag:
+
+```bash
+visor --check all --timeout 300000  # 5 minute global timeout (milliseconds)
+```
+
+This timeout applies to AI operations. The default is 1200000ms (20 minutes).
 
 ## Examples
 
+### Basic Command Timeout
+
 ```yaml
 steps:
   fetch:
     type: command
-    timeout: 120
+    timeout: 120  # 2 minutes
     exec: curl -s https://example.com/data
 
   process:
@@ -41,10 +159,48 @@ steps:
     exec: echo "{{ outputs.fetch }}" | jq '.items | length'
 ```
 
-If `fetch` exceeds 120s, `process` is skipped with reason `dependency_failed`.
+If `fetch` exceeds 120 seconds, `process` is skipped with reason `dependency_failed`.
+
+### Mixed Provider Timeouts
+
+```yaml
+steps:
+  # Command: seconds
+  build:
+    type: command
+    timeout: 300
+    exec: npm run build
+
+  # HTTP: milliseconds
+  notify:
+    type: http_client
+    timeout: 10000
+    url: https://webhook.example.com
+    depends_on: [build]
+
+  # AI: milliseconds (nested)
+  review:
+    type: ai
+    prompt: "Review the build output"
+    ai:
+      timeout: 60000
+    depends_on: [build]
+```
 
 ## Notes
 
-- Be mindful of units when switching providers (seconds vs milliseconds).
-- Consider adding `fail_if` conditions for additional guarding (e.g., `fail_if: !output || length(output.items) === 0`).
+- **Unit awareness**: Be mindful of units when switching between providers (seconds vs milliseconds).
+- **Dependency propagation**: When a check times out, all direct dependents are skipped.
+- **Failure handling**: Consider using [`fail_if`](./fail-if.md) conditions for additional guarding.
+- **Routing**: Use [`on_fail`](./failure-routing.md) to define retry or recovery behavior on timeout.
+
+## Related Documentation
 
+- [Command Provider](./command-provider.md) - Shell command execution
+- [HTTP Integration](./http.md) - HTTP client and webhook providers
+- [MCP Provider](./mcp-provider.md) - MCP tool execution
+- [AI Configuration](./ai-configuration.md) - AI provider settings
+- [Human Input Provider](./human-input-provider.md) - Interactive input
+- [Custom Tools](./custom-tools.md) - YAML-defined tools
+- [Git Checkout Provider](./providers/git-checkout.md) - Repository checkout
+- [Failure Routing](./failure-routing.md) - Handling failures and retries

package/dist/docs/troubleshooting.md

@@ -1,7 +1,177 @@
-## 🛠️ Troubleshooting
+# Troubleshooting
 
-- Increase logging with `--debug` or action `debug: true`.
-- Validate config locally: `npx -y @probelabs/visor@latest --check all`.
-- Ensure required permissions in Actions: contents/read, pull-requests/write, issues/write, checks/write.
-- If annotations don’t appear, confirm schema is `code-review` and issues have file/line.
-- For remote extends, set `--allowed-remote-patterns` or disable via `--no-remote-extends`.
+This guide covers common issues and their solutions when using Visor.
+
+For in-depth debugging techniques including JavaScript expression debugging, Liquid template inspection, and OpenTelemetry tracing, see the [Debugging Guide](./debugging.md).
+
+## Quick Diagnostics
+
+```bash
+# Enable debug mode for detailed output
+visor --check all --debug
+
+# Validate configuration before running
+visor validate --config .visor.yaml
+```
+
+## Common Issues
+
+### Configuration Not Found
+
+**Symptoms**: Visor runs but uses default configuration instead of your custom config.
+
+**Solutions**:
+1. Ensure your config file is named `.visor.yaml` (with leading dot) in the project root
+2. Explicitly specify the config path: `visor --config path/to/.visor.yaml`
+3. Check for YAML syntax errors: `visor validate --config .visor.yaml`
+
+### GitHub Check Runs Not Appearing
+
+**Symptoms**: PR comments work but GitHub Check runs are missing.
+
+**Solutions**:
+1. Verify workflow permissions include `checks: write`:
+   ```yaml
+   permissions:
+     contents: read
+     pull-requests: write
+     issues: write
+     checks: write
+   ```
+2. For fork PRs, check runs require `pull_request_target` trigger - see [Fork PR Support](./GITHUB_CHECKS.md#fork-pr-support)
+3. Ensure `create-check: 'true'` in action inputs (default)
+4. Check action logs for permission errors (403)
+
+### Annotations Not Appearing on Files
+
+**Symptoms**: Check runs exist but no inline annotations on PR files.
+
+**Solutions**:
+1. Confirm the check uses `schema: code-review`
+2. Ensure issues include `file` and `line` properties
+3. Verify the file paths match the PR diff (relative paths from repo root)
+
+### AI Provider Errors
+
+**Symptoms**: Checks fail with API errors or return no results.
+
+**Solutions**:
+1. Verify API key is set correctly:
+   - `GOOGLE_API_KEY` for Gemini
+   - `ANTHROPIC_API_KEY` for Claude
+   - `OPENAI_API_KEY` for OpenAI
+2. Check API key permissions and quotas
+3. Enable debug mode to see full API responses: `--debug`
+4. For rate limiting, reduce `--max-parallelism` or add delays between checks
+
+### Fork PR Permission Errors (403)
+
+**Symptoms**: `Resource not accessible by integration` errors for fork PRs.
+
+**Cause**: GitHub restricts write permissions for workflows triggered by fork PRs.
+
+**Solutions**:
+1. **Accept comment-only mode**: Visor automatically falls back to PR comments
+2. **Enable full fork support**: Use `pull_request_target` trigger (see [GITHUB_CHECKS.md](./GITHUB_CHECKS.md#fork-pr-support))
+
+### Remote Extends Not Loading
+
+**Symptoms**: Configuration fails to load remote URLs in `extends` field.
+
+**Solutions**:
+1. Allow specific URL patterns: `--allowed-remote-patterns "https://github.com/,https://raw.githubusercontent.com/"`
+2. Or disable remote extends entirely: `--no-remote-extends`
+3. Check network connectivity and URL accessibility
+
+### Timeout Errors
+
+**Symptoms**: Checks fail with timeout messages.
+
+**Solutions**:
+1. Increase timeout: `--timeout 300000` (5 minutes in ms)
+2. Reduce check complexity or split into smaller checks
+3. Check for slow network or API responses
+4. See [Timeouts Guide](./timeouts.md) for detailed configuration
+
+### Output Access Issues in Checks
+
+**Symptoms**: `outputs is undefined` or `Cannot read property` errors.
+
+**Solutions**:
+1. Ensure `depends_on` is set for checks that need outputs from other checks:
+   ```yaml
+   steps:
+     my-check:
+       type: command
+       depends_on: [previous-check]  # Required to access outputs
+       exec: echo "{{ outputs['previous-check'] }}"
+   ```
+2. Use optional chaining for safe access: `outputs?.['check-name']?.property`
+3. Debug output structure with the `log()` function - see [Debugging Guide](./debugging.md#debugging-javascript-expressions)
+
+### forEach Not Iterating
+
+**Symptoms**: Check with `forEach: true` runs once instead of iterating.
+
+**Solutions**:
+1. Ensure the output is an array (not a single object)
+2. Use `transform_js` to convert output to array format if needed
+3. Debug with `log()` to inspect the actual output structure
+
+## GitHub Actions Permissions
+
+Required permissions for full functionality:
+
+```yaml
+permissions:
+  contents: read        # Read repository content
+  pull-requests: write  # Post PR comments
+  issues: write         # Post issue comments
+  checks: write         # Create check runs and annotations
+```
+
+## Environment Variables
+
+| Variable | Purpose |
+|----------|---------|
+| `VISOR_TELEMETRY_ENABLED` | Enable OpenTelemetry tracing |
+| `VISOR_TELEMETRY_SINK` | Trace output: `file`, `otlp`, or `console` |
+| `DEBUG=1` or `VERBOSE=1` | Enable verbose internal logging |
+| `VISOR_NOBROWSER=true` | Skip auto-opening browser (CI/headless) |
+
+## Increasing Verbosity
+
+Different verbosity levels for troubleshooting:
+
+```bash
+# Standard verbose mode
+visor --check all --verbose
+
+# Full debug mode (includes AI interactions)
+visor --check all --debug
+
+# Debug mode in GitHub Action
+- uses: probelabs/visor@v1
+  with:
+    debug: 'true'
+```
+
+## Validating Configuration
+
+Always validate your configuration before running:
+
+```bash
+# Validate config syntax and schema
+visor validate --config .visor.yaml
+
+# Test locally with a specific check
+visor --check security --config .visor.yaml --output table
+```
+
+## Further Reading
+
+- [Debugging Guide](./debugging.md) - Comprehensive debugging techniques
+- [Configuration Reference](./configuration.md) - Full configuration options
+- [GitHub Checks Integration](./GITHUB_CHECKS.md) - Check runs and annotations
+- [Telemetry Setup](./telemetry-setup.md) - OpenTelemetry tracing
+- [Timeouts Guide](./timeouts.md) - Timeout configuration

package/dist/docs/workflow-creation-guide.md

@@ -100,6 +100,8 @@ Available events:
 - `issue_opened` - Issue created
 - `issue_comment` - Comment on issue or PR
 - `manual` - CLI execution (no event)
+- `schedule` - Scheduled/cron-triggered execution
+- `webhook_received` - HTTP webhook was received (via http_input provider)
 
 ---
 
@@ -246,7 +248,7 @@ Output structure: `{ text: string, ts: number }`
 
 ### 5. Log Check (`type: log`)
 
-Output messages to the console/log.
+Output messages to the console/log. Useful for debugging workflows and displaying execution information.
 
 ```yaml
 steps:
@@ -261,12 +263,14 @@ steps:
       - {{ item.name }}: {{ item.status }}
       {% endfor %}
 
-    level: info               # info, warn, error, debug
+    level: info               # debug, info, warn, error
     include_pr_context: false
     include_dependencies: false
     include_metadata: false
 ```
 
+Note: The type must be `log` (not `logger`).
+
 ### 6. Script Check (`type: script`)
 
 Execute JavaScript code.
@@ -617,10 +621,63 @@ steps:
     type: ai
     depends_on: [initial-analysis]
     reuse_ai_session: initial-analysis
-    session_mode: clone        # or 'continue'
+    session_mode: clone        # or 'append'
     prompt: "Now look for security issues in what we discussed"
 ```
 
+Session modes:
+- `clone` (default): Copy the conversation history to a new session
+- `append`: Share the conversation history (subsequent messages append to the same session)
+
+### Transform Output (`transform`, `transform_js`)
+
+Transform step outputs before they're consumed by dependent steps:
+
+```yaml
+steps:
+  fetch-data:
+    type: http_client
+    url: "https://api.example.com/data"
+    # Liquid transform
+    transform: |
+      {% assign items = response.data.items %}
+      {{ items | json }}
+    # OR JavaScript transform (alternative)
+    transform_js: |
+      const data = JSON.parse(output);
+      return data.items.filter(i => i.active);
+```
+
+### Init Hook (`on_init`)
+
+Run preprocessing steps before a step executes:
+
+```yaml
+steps:
+  ai-review:
+    type: ai
+    on_init:
+      run:
+        # Invoke a tool to enrich context
+        - tool: fetch-jira
+          with:
+            issue_key: "{{ pr.body | regex_search: '[A-Z]+-[0-9]+' }}"
+          as: jira_context
+        # Or invoke another step
+        - step: enrich-context
+          as: extra_context
+    prompt: |
+      Review the code with this context:
+      JIRA: {{ outputs['jira_context'] | json }}
+```
+
+The `on_init` hook allows:
+- `run`: Array of tool invocations, step invocations, or workflow invocations
+- `run_js`: Dynamic computation of what to run
+- `transitions`: Declarative routing rules
+
+See [on_init Hook RFC](./rfc/on_init-hook.md) for detailed documentation.
+
 ---
 
 ## Testing DSL
@@ -1271,4 +1328,45 @@ visor test --list
 
 # Validate tests only
 visor test --validate
+
+# Simulate event
+visor --config workflow.yaml --event pr_opened
 ```
+
+---
+
+## Related Documentation
+
+For more detailed information on specific topics:
+
+### Core Concepts
+- [Configuration Reference](./configuration.md) - Full configuration options
+- [Event Triggers](./event-triggers.md) - Event types and filtering
+- [Dependencies](./dependencies.md) - Dependency patterns and execution order
+- [Failure Routing](./failure-routing.md) - Retry, remediation, and routing
+
+### Providers
+- [AI Configuration](./ai-configuration.md) - AI provider settings
+- [Claude Code](./claude-code.md) - Claude Code provider details
+- [MCP Provider](./mcp-provider.md) - MCP tool integration
+- [Command Provider](./command-provider.md) - Shell command execution
+- [HTTP Provider](./http.md) - HTTP client and webhooks
+- [Git Checkout](./providers/git-checkout.md) - Repository checkout
+- [Memory](./memory.md) - State persistence
+- [Script](./script.md) - JavaScript execution
+
+### Testing
+- [Testing Getting Started](./testing/getting-started.md) - Quick start for testing
+- [Testing DSL Reference](./testing/dsl-reference.md) - Complete test syntax
+- [Fixtures and Mocks](./testing/fixtures-and-mocks.md) - Test data setup
+- [Assertions](./testing/assertions.md) - Assertion syntax
+
+### Advanced Topics
+- [Workflow Style Guide](./guides/workflow-style-guide.md) - Best practices
+- [Criticality Modes](./guides/criticality-modes.md) - Safety levels
+- [Fault Management](./guides/fault-management-and-contracts.md) - Contracts and guards
+- [on_init Hook](./rfc/on_init-hook.md) - Context preprocessing
+- [Output History](./output-history.md) - Accessing historical outputs
+- [Tag Filtering](./tag-filtering.md) - Selective execution
+- [Liquid Templates](./liquid-templates.md) - Template syntax
+- [Debugging](./debugging.md) - Troubleshooting workflows