@probelabs/visor 0.1.124 → 0.1.126

package/dist/docs/commands.md

@@ -1,8 +1,254 @@
-## 💬 PR Comment Commands
+# Visor Commands
 
-- `/review` – Rerun all configured checks on the pull request
-- `/review --check security` – Run only security checks
-- `/review --check performance` – Run only performance checks
-- `/visor …` – Ask the assistant a question about the code or context
-- `/review --help` – Show available review commands
+This document covers both the CLI commands and PR comment commands available in Visor.
 
+## CLI Commands
+
+The main entry point is the `visor` command. Run `visor --help` to see all available options.
+
+### Main Command
+
+```bash
+visor [options]
+```
+
+Run configured checks against the current repository. Without any options, Visor will discover and load configuration from `.visor.yaml` in the project root.
+
+### Subcommands
+
+#### `visor validate` / `visor lint`
+
+Validate a Visor configuration file without running checks.
+
+```bash
+visor validate --config path/to/config.yaml
+visor lint --config .visor.yaml
+```
+
+#### `visor test`
+
+Run the Visor test framework for testing workflows.
+
+```bash
+visor test [path] [options]
+```
+
+**Options:**
+- `--config <path>` - Path to test suite file, directory, or glob pattern
+- `--only <name>` - Run only tests matching this name (supports `CASE` or `CASE#STAGE`)
+- `--bail` - Stop on first failure
+- `--list` - List discovered tests without running them
+- `--validate` - Validate test files without running them
+- `--json <path>` - Write JSON report to path (use `-` for stdout)
+- `--report junit:<path>` - Write JUnit XML report
+- `--summary md:<path>` - Write Markdown summary
+- `--max-parallel <n>` - Maximum parallel test cases per suite
+- `--max-suites <n>` - Maximum parallel test suites
+- `--no-mocks` - Disable mocks
+
+**Examples:**
+```bash
+visor test tests/                           # Run all tests in directory
+visor test --config tests/my-suite.yaml     # Run specific test suite
+visor test --only "my test case"            # Run specific test
+visor test --bail --json results.json       # Stop on failure, save JSON report
+```
+
+#### `visor code-review` / `visor review`
+
+Run the built-in code review workflow against the current repository.
+
+```bash
+visor code-review [options]
+visor review [options]
+```
+
+This uses the default code-review configuration from `defaults/code-review.yaml`.
+
+#### `visor build`
+
+Run the agent builder workflow to create or modify Visor agents.
+
+```bash
+visor build <path/to/agent.yaml> [options]
+```
+
+**Example:**
+```bash
+visor build agents/my-agent.yaml --message "Add error handling"
+```
+
+#### `visor mcp-server`
+
+Start Visor as an MCP (Model Context Protocol) server.
+
+```bash
+visor mcp-server [options]
+```
+
+**Options:**
+- `--config <path>` - Configuration file path
+- `--mcp-tool-name <name>` - Custom tool name for the MCP server
+- `--mcp-tool-description <desc>` - Custom tool description
+
+### Common CLI Options
+
+#### Check Selection
+- `-c, --check <type>` - Specify check type (can be used multiple times)
+
+#### Output Options
+- `-o, --output <format>` - Output format: `table` (default), `json`, `markdown`, `sarif`
+- `--output-file <path>` - Write output to file instead of stdout
+
+#### Configuration
+- `--config <path>` - Path to configuration file
+
+#### Execution Control
+- `--timeout <ms>` - Timeout for operations (default: 1200000ms / 20 minutes)
+- `--max-parallelism <count>` - Maximum parallel checks (default: 3)
+- `--fail-fast` - Stop execution on first failure
+
+#### Tag Filtering
+- `--tags <tags>` - Include checks with these tags (comma-separated)
+- `--exclude-tags <tags>` - Exclude checks with these tags (comma-separated)
+
+See [Tag Filtering](tag-filtering.md) for detailed tag filtering documentation.
+
+#### Verbosity
+- `--debug` - Enable debug mode with detailed output
+- `-v, --verbose` - Increase verbosity (without full debug)
+- `-q, --quiet` - Reduce verbosity to warnings and errors
+
+#### Code Context
+- `--enable-code-context` - Force include code diffs in analysis
+- `--disable-code-context` - Force exclude code diffs from analysis
+- `--analyze-branch-diff` - Analyze diff vs base branch (auto-enabled for code-review schemas)
+
+#### Event Simulation
+- `--event <type>` - Simulate GitHub event: `pr_opened`, `pr_updated`, `issue_opened`, `issue_comment`, `manual`, `all`
+
+#### Interactive Mode
+- `--tui` - Enable interactive TUI (chat + logs tabs)
+- `--message <text>` - Message for human-input checks (inline text or file path)
+
+#### Debug Server
+- `--debug-server` - Start debug visualizer server for live execution visualization
+- `--debug-port <port>` - Port for debug server (default: 3456)
+
+#### Workspace Options
+- `--keep-workspace` - Keep workspace folders after execution (for debugging)
+- `--workspace-path <path>` - Workspace base path
+- `--workspace-here` - Place workspace under current directory
+- `--workspace-name <name>` - Workspace directory name
+- `--workspace-project-name <name>` - Main project folder name inside workspace
+
+#### Other Options
+- `--slack` - Enable Slack Socket Mode runner
+- `--mode <mode>` - Run mode: `cli` (default) or `github-actions`
+- `--no-remote-extends` - Disable loading configurations from remote URLs
+- `--allowed-remote-patterns <patterns>` - Comma-separated list of allowed URL prefixes for remote config extends
+
+### Examples
+
+```bash
+# Run all configured checks
+visor
+
+# Run specific check types
+visor --check security --check performance
+
+# Output as JSON to file
+visor --output json --output-file results.json
+
+# Run with 5 minute timeout and 5 parallel checks
+visor --timeout 300000 --max-parallelism 5
+
+# Run only checks tagged as 'local' or 'fast'
+visor --tags local,fast
+
+# Run security checks but skip slow ones
+visor --tags security --exclude-tags slow
+
+# Enable debug mode with markdown output
+visor --debug --output markdown
+
+# Generate SARIF report
+visor --output sarif > results.sarif
+
+# Interactive TUI mode
+visor --tui
+
+# Debug visualizer
+visor --debug-server --debug-port 3456
+```
+
+---
+
+## PR Comment Commands
+
+When Visor is configured as a GitHub Action, it responds to slash commands in PR comments.
+
+### Built-in Commands
+
+#### `/help`
+
+Show available commands and their descriptions.
+
+```
+/help
+```
+
+Displays a help message listing all configured custom commands plus built-in commands.
+
+#### `/status`
+
+Show current PR status and metrics.
+
+```
+/status
+```
+
+Displays information about the current state of the pull request.
+
+### Custom Commands
+
+Custom commands are configured via the `command` property in check definitions within your `.visor.yaml` file.
+
+**Example configuration:**
+
+```yaml
+checks:
+  security-review:
+    type: ai
+    command: review-security
+    prompt: "Review this code for security issues"
+
+  performance-check:
+    type: ai
+    command: check-perf
+    prompt: "Analyze performance implications"
+```
+
+With the above configuration, the following commands become available in PR comments:
+
+- `/review-security` - Runs the security-review check
+- `/check-perf` - Runs the performance-check check
+
+When a user comments `/help` on a PR, they'll see these custom commands listed along with which checks they trigger.
+
+### Command Behavior
+
+- Commands are **case-insensitive** (`/Help` and `/help` both work)
+- Commands must start with a forward slash (`/`)
+- Unrecognized commands are ignored
+- Commands can only be used in PR comments when Visor is running as a GitHub Action
+
+---
+
+## Related Documentation
+
+- [Configuration](configuration.md) - Full configuration reference
+- [Tag Filtering](tag-filtering.md) - Filter checks by tags
+- [CI/CLI Mode](ci-cli-mode.md) - Running Visor in CI environments
+- [Debug Visualizer](debug-visualizer.md) - Interactive debugging
+- [Event Triggers](event-triggers.md) - Configuring when checks run

package/dist/docs/configuration.md

@@ -21,11 +21,97 @@ visor validate --config examples/enhanced-config.yaml
 
 The `validate` command checks for:
 - **Missing required fields** (e.g., `version`)
-- **Invalid check types** (must be: `ai`, `claude-code`, `command`, `http`, `http_input`, `http_client`, `noop`, `log`, `github`)
+- **Invalid check types** (see [Check Types](#check-types) below)
 - **Invalid event triggers** (e.g., `scheduled` should be `schedule`)
 - **Incorrect field names** and typos
 - **Schema compliance** for all configuration options
 
+### Check Types
+
+Visor supports the following check types:
+
+| Type | Description | Documentation |
+|------|-------------|---------------|
+| `ai` | AI-powered analysis using LLMs | [AI Configuration](./ai-configuration.md) |
+| `claude-code` | Claude Code SDK integration | [Claude Code](./claude-code.md) |
+| `command` | Execute shell commands | [Command Provider](./command-provider.md) |
+| `script` | Custom JavaScript logic | [Script](./script.md) |
+| `http` | Send HTTP requests (output) | [HTTP Integration](./http.md) |
+| `http_input` | Receive webhooks | [HTTP Integration](./http.md) |
+| `http_client` | Fetch data from APIs | [HTTP Integration](./http.md) |
+| `mcp` | MCP tool execution | [MCP Provider](./mcp-provider.md) |
+| `memory` | Key-value storage operations | [Memory](./memory.md) |
+| `workflow` | Reusable workflow invocation | [Workflows](./workflows.md) |
+| `git-checkout` | Git repository checkout | [Git Checkout](./providers/git-checkout.md) |
+| `human-input` | Request user input | [Human Input](./human-input-provider.md) |
+| `github` | GitHub API operations | See [Native GitHub Provider](#native-github-provider) |
+| `log` | Debug logging | [Debugging](./debugging.md) |
+| `noop` | No-operation (for routing) | Used for control flow |
+
+### Criticality and Contracts
+
+Steps can declare their operational criticality, which drives default safety policies for contracts, retries, and loop budgets. See the [Criticality Modes Guide](./guides/criticality-modes.md) for complete documentation.
+
+```yaml
+steps:
+  post-comment:
+    type: github
+    criticality: external   # external | internal | policy | info
+    op: comment.create
+
+    # Preconditions - must hold before execution
+    assume:
+      - "outputs['permission-check'].allowed === true"
+      - "env.DRY_RUN !== 'true'"
+
+    # Postconditions - assertions about produced output
+    guarantee:
+      - "output && typeof output.id === 'number'"
+
+    # Other step configuration...
+```
+
+#### Criticality Levels
+
+| Level | Description | Use When |
+|-------|-------------|----------|
+| `external` | Mutates external systems (GitHub, HTTP POST, etc.) | Step has side effects outside the engine |
+| `internal` | Steers execution (forEach, routing, flags) | Step controls workflow routing |
+| `policy` | Enforces permissions/compliance | Step gates external actions |
+| `info` | Read-only, non-critical | Pure computation, safe to fail |
+
+#### Contracts: assume and guarantee
+
+- **`assume`**: Preconditions that must hold before execution. If false, the step is skipped (skipReason='assume').
+- **`guarantee`**: Postconditions about the produced output. Violations are recorded as error issues with ruleId "contract/guarantee_failed".
+
+```yaml
+steps:
+  critical-step:
+    type: http
+    criticality: external
+    url: "https://api.example.com/deploy"
+    method: POST
+
+    # Only run if authenticated and not in dry-run mode
+    assume:
+      - "env.API_TOKEN"
+      - "env.DRY_RUN !== 'true'"
+
+    # Verify the response is valid
+    guarantee:
+      - "output && output.status === 'success'"
+      - "output.deployment_id !== undefined"
+
+    schema: plain
+```
+
+**Best Practices:**
+- Use `assume` for pre-execution prerequisites (env/memory/upstream), not for checking this step's output
+- Use `guarantee` for assertions about this step's produced output (shape, values, invariants)
+- Use `fail_if` for policy/threshold decisions
+- Keep expressions deterministic (no time/random/network)
+
 Example validation output:
 ```
 🔍 Visor Configuration Validator
@@ -50,7 +136,7 @@ If there are errors, you'll get detailed messages with hints:
 ```
 ❌ Configuration validation failed!
 
-Error: Invalid check type "webhook". Must be: ai, claude-code, command, http, http_input, http_client, noop, log, github
+Error: Invalid check type "webhook". Must be: ai, claude-code, mcp, command, script, http, http_input, http_client, memory, noop, log, github, human-input, workflow, git-checkout
 
 💡 Hint: The 'webhook' type has been renamed to 'http' for output and 'http_input' for input.
 ```
@@ -63,7 +149,6 @@ Override global AI settings at the check level:
 # Global AI settings (optional)
 ai_provider: anthropic  # or google, openai, bedrock
 ai_model: claude-3-sonnet
-ai_temperature: 0.2
 
 steps:
   performance-review:
@@ -71,7 +156,6 @@ steps:
     ai:
       provider: google
       model: gemini-1.5-pro
-      temperature: 0.1
     prompt: "Analyze performance metrics and provide optimization suggestions"
 
   security-review:
@@ -322,3 +406,53 @@ Notes:
 - Requires `GITHUB_TOKEN` (or `github-token` Action input) and `GITHUB_REPOSITORY` in environment.
 - Use Liquid `safe_label` / `safe_label_list` to constrain labels to `[A-Za-z0-9:/\- ]` (alphanumerics, colon, slash, hyphen, and space).
 - Provider errors surface as issues (e.g., `github/missing_token`, `github/op_failed`) and won't abort the whole run.
+
+### Additional Configuration Options
+
+The following global configuration options are available and documented in detail in their respective guides:
+
+| Option | Description | Documentation |
+|--------|-------------|---------------|
+| `max_parallelism` | Maximum number of checks to run in parallel (default: 3) | [Performance](./performance.md) |
+| `fail_fast` | Stop execution when any check fails (default: false) | [Performance](./performance.md) |
+| `fail_if` | Global failure condition expression | [Fail If](./fail-if.md) |
+| `tag_filter` | Filter checks by tags (include/exclude) | [Tag Filtering](./tag-filtering.md) |
+| `routing` | Global routing defaults for retry/goto policies | [Failure Routing](./failure-routing.md) |
+| `limits` | Global execution limits (max_runs_per_check, max_workflow_depth) | [Limits](./limits.md) |
+| `tools` | Custom tool definitions for MCP blocks | [Custom Tools](./custom-tools.md) |
+| `imports` | Import workflow definitions from external files | [Workflows](./workflows.md) |
+| `inputs`/`outputs` | Workflow input/output definitions | [Workflows](./workflows.md) |
+| `http_server` | HTTP server for receiving webhooks | [HTTP Integration](./http.md) |
+| `memory` | Memory storage configuration | [Memory](./memory.md) |
+| `output` | Output configuration (PR comments, file comments) | [Output Formats](./output-formats.md) |
+| `workspace` | Workspace isolation configuration | [Workspace Isolation RFC](./rfc/workspace-isolation.md) |
+
+Example combining several options:
+
+```yaml
+version: "1.0"
+
+max_parallelism: 5
+fail_fast: true
+
+tag_filter:
+  include: [security, performance]
+  exclude: [experimental]
+
+limits:
+  max_runs_per_check: 50
+  max_workflow_depth: 3
+
+routing:
+  max_loops: 10
+  defaults:
+    on_fail:
+      retry:
+        max: 2
+        backoff:
+          mode: exponential
+          delay_ms: 1000
+
+steps:
+  # ... your step definitions
+```