@probelabs/visor 0.1.106 → 0.1.107

package/dist/examples/README.md

@@ -0,0 +1,384 @@
+# Visor Examples
+
+This directory contains SDK examples (programmatic usage) and configuration examples (YAML files).
+
+## 🚀 SDK Examples (Programmatic Usage)
+
+Use Visor from Node.js without shelling out.
+
+### Running SDK Examples
+
+```bash
+# Build SDK first
+npm run build:sdk
+
+# Basic SDK examples
+node examples/sdk-basic.mjs           # Minimal (5 lines)
+node examples/sdk-cjs.cjs             # CommonJS
+node examples/sdk-manual-config.mjs   # Manual config
+node examples/sdk-comprehensive.mjs   # Complex with dependencies
+
+# TypeScript (full type safety with SDK types)
+npx tsc examples/sdk-typescript.ts --module esnext --target es2022 --moduleResolution bundler --esModuleInterop --skipLibCheck && node examples/sdk-typescript.js
+```
+
+### 🧮 Human-Input Calculator Examples
+
+Interactive calculator demonstrating human-in-the-loop workflows:
+
+```bash
+# Interactive calculator with table output
+bun examples/calculator-sdk-real.ts
+
+# Calculator with JSON output (programmatic processing)
+bun examples/calculator-sdk-json.ts
+
+# Fully automated calculator (for testing/automation)
+bun examples/calculator-sdk-automated.ts
+bun examples/calculator-sdk-automated.ts 10 5 +  # With args
+
+# YAML config version
+./dist/index.js --config examples/calculator-config.yaml --message "10" --check get-number1
+```
+
+**Features demonstrated:**
+- ✅ Human input via `human-input` check type
+- ✅ Memory provider for state management
+- ✅ JavaScript execution in memory provider
+- ✅ Dependency chains with fail_if validation
+- ✅ JSON output for programmatic processing
+- ✅ Custom visualization in SDK scripts
+- ✅ Suppressing stdout/stderr for clean JSON responses
+
+See `docs/sdk.md` for full SDK documentation.
+
+---
+
+## 📝 Configuration Examples (YAML)
+
+Example configurations demonstrating various Visor features and use cases.
+
+## 📁 Files Overview
+
+### Basic Examples
+- **`quick-start-tags.yaml`** - Simple configuration showing basic tag usage
+- **`visor-with-tags.yaml`** - Comprehensive configuration with all tag features
+ - **`routing-basic.yaml`** - Failure routing with retry + goto ancestor
+ - **`routing-on-success.yaml`** - on_success post-steps + single jump-back
+ - **`routing-foreach.yaml`** - forEach remediation with run + retry
+ - **`routing-dynamic-js.yaml`** - Dynamic routing via goto_js/run_js
+
+### GitHub Actions Workflows
+- **`github-workflow-with-tags.yml`** - Progressive code review workflow using tags
+
+### Environment Configurations
+- **`environments/visor.base.yaml`** - Base configuration with all check definitions
+- **`environments/visor.dev.yaml`** - Development environment (fast, local checks)
+- **`environments/visor.staging.yaml`** - Staging environment (balanced checks)
+- **`environments/visor.prod.yaml`** - Production environment (comprehensive validation)
+
+## 🚀 Quick Start
+
+### 1. Basic Tag Usage
+
+Start with the simple configuration:
+
+```bash
+# Copy the quick-start example
+cp examples/quick-start-tags.yaml .visor.yaml
+
+# Run local checks
+visor --tags local,fast
+
+# Run comprehensive checks
+visor --tags remote,comprehensive
+```
+
+### 2. Environment-Based Configuration
+
+Use different configurations for different environments:
+
+```bash
+# Development
+visor --config examples/environments/visor.dev.yaml
+
+# Staging
+visor --config examples/environments/visor.staging.yaml
+
+# Production
+visor --config examples/environments/visor.prod.yaml
+```
+
+### 3. GitHub Actions Integration
+
+Copy the workflow to your repository:
+
+```bash
+cp examples/github-workflow-with-tags.yml .github/workflows/code-review.yml
+```
+
+## 🏷️ Tag Strategy Guide
+
+### Recommended Tag Taxonomy
+
+#### Environment Tags
+- `local` - Runs on developer machines
+- `remote` - Runs in CI/CD
+- `dev` - Development environment
+- `staging` - Staging environment
+- `prod` - Production environment
+
+#### Speed Tags
+- `fast` - Completes in < 30 seconds
+- `slow` - Takes > 30 seconds
+- `comprehensive` - Thorough but time-consuming
+
+#### Category Tags
+- `security` - Security-related checks
+- `performance` - Performance analysis
+- `quality` - Code quality and style
+- `testing` - Test-related checks
+- `documentation` - Documentation checks
+
+#### Priority Tags
+- `critical` - Must pass for deployment
+- `optional` - Nice to have but not blocking
+- `experimental` - Beta features
+
+### Tag Combination Examples
+
+```bash
+# Fast security checks for local development
+visor --tags local,fast,security
+
+# All critical checks for production
+visor --tags prod,critical
+
+# Comprehensive review excluding experimental
+visor --tags comprehensive --exclude-tags experimental
+
+# Just the essentials
+visor --tags critical,fast
+```
+
+## 📊 Execution Profiles
+
+### Profile 1: Developer (Local)
+```yaml
+tag_filter:
+  include: ["local", "fast"]
+  exclude: ["slow", "experimental"]
+```
+- **Goal**: Quick feedback during development
+- **Runtime**: < 1 minute
+- **Use Case**: Pre-commit hooks, local testing
+
+### Profile 2: Pull Request (CI)
+```yaml
+tag_filter:
+  include: ["remote", "critical"]
+  exclude: ["experimental"]
+```
+- **Goal**: Validate changes before merge
+- **Runtime**: 2-5 minutes
+- **Use Case**: GitHub Actions on PR
+
+### Profile 3: Pre-Production (Staging)
+```yaml
+tag_filter:
+  include: ["staging", "comprehensive"]
+  exclude: ["experimental", "optional"]
+```
+- **Goal**: Thorough validation before production
+- **Runtime**: 5-10 minutes
+- **Use Case**: Staging deployment pipeline
+
+### Profile 4: Production Release
+```yaml
+tag_filter:
+  include: ["prod", "critical", "comprehensive"]
+  exclude: ["experimental"]
+```
+- **Goal**: Maximum confidence for production
+- **Runtime**: 10+ minutes
+- **Use Case**: Production deployment gate
+
+## 🔧 Advanced Patterns
+
+### Pattern 1: Progressive Enhancement
+
+Start with fast checks and progressively run more comprehensive ones:
+
+```yaml
+# Stage 1: Critical issues (fail fast)
+visor --tags critical,fast --fail-fast
+
+# Stage 2: Security scan (if stage 1 passes)
+visor --tags security --exclude-tags fast
+
+# Stage 3: Comprehensive review (if all pass)
+visor --tags comprehensive --exclude-tags security,critical
+```
+
+### Pattern 2: Conditional Execution
+
+Run checks based on file changes:
+
+```yaml
+steps:
+  frontend-checks:
+    tags: ["frontend", "conditional"]
+    on: [pr_opened]
+    if: "filesChanged.some(f => f.endsWith('.tsx'))"
+
+  backend-checks:
+    tags: ["backend", "conditional"]
+    on: [pr_opened]
+    if: "filesChanged.some(f => f.endsWith('.py'))"
+```
+
+### Pattern 3: Dependency Chains with Tags
+
+```yaml
+steps:
+  quick-scan:
+    tags: ["local", "fast"]
+
+  deep-scan:
+    tags: ["remote", "slow"]
+    depends_on: [quick-scan]  # Only if quick-scan is included
+
+  report:
+    tags: ["reporting"]
+    depends_on: [quick-scan, deep-scan]  # Uses whatever ran
+```
+
+## 🎯 Best Practices
+
+1. **Start Simple**: Begin with `local`/`remote` or `fast`/`slow`
+2. **Be Consistent**: Use the same tags across all projects
+3. **Document Tags**: Maintain a tag glossary in your docs
+4. **Review Regularly**: Audit and update tags as needs change
+5. **Measure Impact**: Track execution times and adjust tags accordingly
+
+## 💡 Tips
+
+- Use `visor --help` to see all available options
+- Combine `--tags` and `--exclude-tags` for precise control
+- Set default `tag_filter` in config to avoid repetition
+- Use environment-specific configs with `extends` for DRY principles
+- Test tag filters with `--debug` to see which checks run
+
+## 📚 Further Reading
+
+- [Main README](../README.md) - Complete Visor documentation
+- [Configuration Guide](../docs/configuration.md) - Detailed config options
+- [GitHub Actions Guide](../docs/github-actions.md) - CI/CD integration
+### 4. Human Input Examples
+
+Interactive workflows with human-in-the-loop:
+
+```bash
+# Basic human input patterns
+visor --config examples/human-input-example.yaml
+
+# Interactive calculator (demonstrates memory + JS + human input)
+visor --config examples/calculator-config.yaml
+
+# Run with inline message
+visor --config examples/human-input-example.yaml --check approval-gate --message "yes"
+
+# Run with file input (auto-detected)
+echo "yes" > approval.txt
+visor --config examples/human-input-example.yaml --check approval-gate --message approval.txt
+
+# Run with piped input
+echo "yes" | visor --config examples/human-input-example.yaml --check approval-gate
+```
+
+**Calculator Example:**
+The calculator demonstrates a complete workflow:
+1. Prompts for first number
+2. Prompts for second number
+3. Prompts for operation (+, -, *, /)
+4. Stores values in memory
+5. Calculates result using JavaScript
+6. Displays formatted result
+
+**SDK Usage:**
+
+Two SDK examples are provided:
+
+1. **`calculator-sdk-real.ts`** - Complete, runnable SDK example:
+   - Real imports from Visor SDK
+   - Config defined inline (no YAML needed)
+   - Custom readline-based input hook
+   - Full CheckExecutionEngine usage
+   - Works in interactive or automated mode
+
+   ```bash
+   # Interactive mode
+   ts-node examples/calculator-sdk-real.ts
+
+   # Automated mode (for testing)
+   ts-node examples/calculator-sdk-real.ts 42 7 +
+   ```
+
+2. **`calculator-sdk-example.ts`** - Documentation/template example:
+   - Shows the structure and patterns
+   - Includes comments and explanations
+   - Generates YAML config for CLI usage
+
+**SDK Pattern:**
+```typescript
+import { HumanInputCheckProvider } from '../src/providers/human-input-check-provider';
+import { CheckExecutionEngine } from '../src/check-execution-engine';
+import { VisorConfig } from '../src/types/config';
+
+// Define config inline
+const config: VisorConfig = {
+  version: "1.0",
+  checks: {
+    "my-check": {
+      type: "human-input",
+      prompt: "Enter value:"
+    }
+  },
+  output: { pr_comment: { format: "markdown", group_by: "check", collapse: false } }
+};
+
+// Set custom hook
+HumanInputCheckProvider.setHooks({
+  onHumanInput: async (request) => {
+    return await myCustomHandler(request);
+  }
+});
+
+// Run checks
+const engine = new CheckExecutionEngine();
+const results = await engine.executeChecks(prInfo, config, Object.keys(config.checks));
+```
+
+### 5. Failure Routing Examples
+
+Run the examples directly from the repo root:
+
+```bash
+# Basic retry + goto ancestor
+npx -y @probelabs/visor@latest --config examples/routing-basic.yaml --output table
+
+# on_success: run notify and jump back once to re-run unit-tests
+npx -y @probelabs/visor@latest --config examples/routing-on-success.yaml --output table
+
+# forEach remediation: mark missing items then retry
+npx -y @probelabs/visor@latest --config examples/routing-foreach.yaml --output table
+
+# Dynamic routing with *_js hooks
+npx -y @probelabs/visor@latest --config examples/routing-dynamic-js.yaml --output table
+```
+
+Notes:
+- These examples create small temporary files in the repo (prefixed with `.visor_demo_`).
+  Run `git clean -fdx` or delete the files manually to reset.
+- The `routing` block supports `max_loops` and default retry policies; step-level settings override defaults.
+- See `docs/failure-routing-rfc.md` for full semantics.

package/dist/examples/ai-retry-fallback-config.yaml

@@ -0,0 +1,180 @@
+# Example configuration demonstrating AI retry and fallback mechanisms
+# This shows how to configure automatic retries with exponential backoff
+# and fallback to alternative AI providers when the primary provider fails
+
+version: '1.0'
+
+# Global AI provider settings with retry and fallback
+# Note: Check-level settings (in steps.<name>.ai) override these global settings
+ai_provider: anthropic
+ai_model: claude-3-5-sonnet-20241022
+
+steps:
+  # Example 1: Basic retry configuration
+  # Retries the same provider with exponential backoff
+  code-review:
+    type: ai
+    prompt: |
+      Review this code for security vulnerabilities and best practices.
+      Focus on:
+      - Authentication and authorization issues
+      - SQL injection and XSS vulnerabilities
+      - Sensitive data exposure
+    schema: code-review
+    on:
+      - pr_opened
+      - pr_updated
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      # Retry configuration for transient failures
+      retry:
+        maxRetries: 3              # Retry up to 3 times (0-50 allowed)
+        initialDelay: 1000         # Start with 1 second delay (0-60000ms)
+        maxDelay: 30000            # Cap delay at 30 seconds (0-300000ms)
+        backoffFactor: 2           # Double delay each retry (1-10 allowed)
+        retryableErrors:           # Optional: custom error patterns to retry on
+          - "rate limit"
+          - "503"
+          - "timeout"
+
+  # Example 2: Automatic fallback with environment detection
+  # Automatically falls back to available providers based on env vars
+  security-scan:
+    type: ai
+    prompt: |
+      Perform a comprehensive security analysis of the changes.
+      Identify critical security vulnerabilities.
+    schema: code-review
+    on:
+      - pr_opened
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      retry:
+        maxRetries: 2
+        initialDelay: 1000
+        backoffFactor: 2
+      # Automatic fallback to any available provider
+      fallback:
+        auto: true                 # Enable automatic provider detection
+        strategy: any              # Try any available provider
+        maxTotalAttempts: 10       # Total attempts across all providers
+
+  # Example 3: Custom fallback provider chain
+  # Explicitly specify fallback providers in order
+  performance-review:
+    type: ai
+    prompt: |
+      Analyze code for performance issues and optimization opportunities.
+    schema: code-review
+    on:
+      - pr_opened
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      retry:
+        maxRetries: 2
+        initialDelay: 500
+        backoffFactor: 2
+      # Custom fallback chain
+      fallback:
+        strategy: custom
+        maxTotalAttempts: 8
+        providers:
+          # First fallback: Try OpenAI GPT-4
+          - provider: openai
+            model: gpt-4-turbo-preview
+            maxRetries: 2           # Per-provider retry override
+            apiKey: "${OPENAI_API_KEY}"  # Can reference env vars
+
+          # Second fallback: Try Google Gemini
+          - provider: google
+            model: gemini-2.0-flash-exp
+            maxRetries: 2
+            apiKey: "${GOOGLE_API_KEY}"
+
+          # Third fallback: AWS Bedrock (requires AWS credentials)
+          - provider: bedrock
+            model: anthropic.claude-3-5-sonnet-20241022-v2:0
+            maxRetries: 1
+            region: us-east-1
+            accessKeyId: "${AWS_ACCESS_KEY_ID}"
+            secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
+
+  # Example 4: Same-model fallback strategy
+  # Falls back to same model on different providers
+  style-review:
+    type: ai
+    prompt: |
+      Review code style and consistency with project conventions.
+    schema: code-review
+    on:
+      - pr_updated
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      retry:
+        maxRetries: 1
+      fallback:
+        strategy: same-model      # Try to use same model on other providers
+        auto: true
+        maxTotalAttempts: 5
+
+  # Example 5: Same-provider fallback strategy
+  # Falls back to different models from same provider
+  documentation-review:
+    type: ai
+    prompt: |
+      Review documentation for completeness and clarity.
+    schema: code-review
+    on:
+      - pr_opened
+    ai:
+      provider: anthropic
+      model: claude-3-5-sonnet-20241022
+      retry:
+        maxRetries: 2
+      fallback:
+        strategy: same-provider    # Try other models from Anthropic
+        providers:
+          - provider: anthropic
+            model: claude-3-opus-20240229
+            maxRetries: 1
+          - provider: anthropic
+            model: claude-3-haiku-20240307
+            maxRetries: 1
+
+  # Example 6: Conservative retry with no fallback
+  # Useful when you want to fail fast and don't want to try alternatives
+  critical-security-check:
+    type: ai
+    prompt: |
+      Perform critical security validation that requires highest accuracy.
+    schema: code-review
+    on:
+      - pr_opened
+    ai:
+      provider: anthropic
+      model: claude-3-opus-20240229  # Use most capable model
+      retry:
+        maxRetries: 1               # Only retry once
+        initialDelay: 2000
+      # No fallback - fail if primary provider fails after retries
+
+output:
+  pr_comment:
+    format: markdown
+    group_by: check
+    collapse: false
+  github_checks:
+    enabled: true
+    per_check: true
+
+# Notes:
+# - Retry logic uses exponential backoff with random jitter to avoid thundering herd
+# - Retries automatically detect transient failures (rate limits, timeouts, 503, etc.)
+# - Fallback providers are tried in order after primary provider exhausts retries
+# - Environment variables are automatically used when fallback.auto is enabled
+# - Each provider in fallback chain can have its own retry configuration
+# - maxTotalAttempts caps total attempts across all providers to prevent excessive API usage

package/dist/examples/ai-with-bash.yaml

@@ -0,0 +1,126 @@
+# Example Visor configuration demonstrating bash command execution in AI checks
+version: "1.0"
+
+# Global AI provider configuration
+ai_provider: anthropic
+ai_model: claude-3-sonnet
+
+steps:
+  # Example 1: Simple - Enable bash with default safe commands
+  git-status-analysis:
+    type: ai
+    prompt: |
+      Analyze the current git repository status:
+      - Check for uncommitted changes
+      - Review the current branch
+      - List recent commits
+      - Identify any potential issues
+    ai:
+      provider: anthropic
+      model: claude-3-opus
+      allowBash: true  # Simple one-line enable
+    on: ["pr_opened", "pr_updated"]
+    tags: ["git", "analysis"]
+
+  # Example 2: Advanced - Custom allow/deny lists for npm commands
+  npm-audit-check:
+    type: ai
+    prompt: |
+      Run npm audit and analyze the security vulnerabilities:
+      - Check for high/critical vulnerabilities
+      - Review outdated dependencies
+      - Suggest remediation steps
+    ai:
+      provider: google
+      model: gemini-2.0-flash-exp
+      allowBash: true
+      bashConfig:
+        allow:
+          - 'npm audit --json'
+          - 'npm outdated --json'
+          - 'npm list --depth=0'
+        timeout: 60000  # 60 second timeout
+    on: ["pr_opened"]
+    tags: ["security", "npm"]
+
+  # Example 3: Advanced - Test execution with custom config
+  test-runner-analysis:
+    type: ai
+    prompt: |
+      Run the test suite and analyze the results:
+      - Execute all tests
+      - Identify failing tests
+      - Review code coverage
+      - Suggest improvements
+    ai:
+      provider: anthropic
+      allowBash: true
+      bashConfig:
+        allow:
+          - 'npm test'
+          - 'npm run test:coverage'
+        deny:
+          - 'npm install'  # Explicitly block installation
+        timeout: 300000  # 5 minute timeout for tests
+        workingDirectory: '.'
+    on: ["pr_opened", "pr_updated"]
+    tags: ["tests", "coverage"]
+
+  # Example 4: Advanced - Build and lint with timeouts
+  build-lint-check:
+    type: ai
+    prompt: |
+      Run build and lint checks:
+      - Execute the build process
+      - Run ESLint
+      - Check TypeScript compilation
+      - Review any errors or warnings
+    ai:
+      provider: openai
+      model: gpt-4
+      allowBash: true
+      bashConfig:
+        allow:
+          - 'npm run build'
+          - 'npm run lint'
+          - 'tsc --noEmit'
+        timeout: 180000  # 3 minute timeout
+    on: ["pr_opened"]
+    tags: ["build", "lint"]
+
+  # Example 5: Expert - Custom commands only (no defaults)
+  custom-commands-only:
+    type: ai
+    prompt: "Run custom analysis commands with strict control"
+    ai:
+      provider: anthropic
+      allowBash: true
+      bashConfig:
+        noDefaultAllow: true  # Disable default safe commands
+        noDefaultDeny: false  # Keep dangerous command blocklist
+        allow:
+          - 'custom-tool analyze'
+          - 'custom-tool report'
+        timeout: 30000
+    on: ["manual"]
+    tags: ["custom", "advanced"]
+
+  # Example 6: Simple - File system analysis with defaults
+  filesystem-analysis:
+    type: ai
+    prompt: |
+      Analyze the project file structure:
+      - List all source files
+      - Check file sizes
+      - Review directory structure
+      - Identify any organizational issues
+    ai:
+      provider: anthropic
+      allowBash: true  # Uses default safe commands (ls, find, etc.)
+    on: ["pr_opened"]
+    tags: ["filesystem", "structure"]
+
+output:
+  pr_comment:
+    enabled: true
+    group_by: check