@probelabs/visor 0.1.106 → 0.1.111

package/dist/docs/fail-if.md

@@ -0,0 +1,95 @@
+# Fail If: Turning conditions into failures
+
+Visor lets you declare simple expressions that fail a check when they evaluate to true. This works for any provider (ai, command, http_*, etc.) and is evaluated as part of the execution engine so dependents can be skipped reliably.
+
+## Where to configure
+
+- Per check:
+
+```yaml
+steps:
+  analyze-bug:
+    type: ai
+    schema: ./schemas/ticket-analysis.json
+    fail_if: output.error
+```
+
+- Global (applies to all checks unless the check overrides with its own `fail_if`):
+
+```yaml
+fail_if: outputs["fetch-tickets"].error
+```
+
+## Evaluation context
+
+Inside the expression you can use:
+
+- `output`: the current check's structured output.
+  - Includes `issues` and any other fields produced by the provider.
+  - For custom schemas, all top‑level JSON fields are preserved and exposed here.
+  - For command output that is JSON, fields are available directly; for plain text, treat `output` as a string.
+- `outputs`: a map of dependency outputs keyed by check name. Each value is that check's `output` if present; otherwise the whole check result.
+
+**Helper functions:**
+- String: `contains(haystack, needle)`, `startsWith(s,prefix)`, `endsWith(s,suffix)`, `length(x)`
+- Control: `always()`, `success()`, `failure()`
+- Author permissions: `hasMinPermission(level)`, `isOwner()`, `isMember()`, `isCollaborator()`, `isContributor()`, `isFirstTimer()` - See [Author Permissions](./author-permissions.md)
+- Issue/file matching helpers (see source FailureConditionEvaluator for the full list)
+
+Truthiness rules follow JavaScript: non‑empty strings are truthy, `""` and `null`/`undefined` are falsy, `0` is falsy.
+
+## What happens when a condition is met
+
+- The engine adds an error‑severity issue to the check with ruleId `<checkName>_fail_if` (or `global_fail_if` for the global rule).
+- Direct dependents of that check are skipped with reason `dependency_failed`.
+- Skipped checks do not execute their providers; they appear as ⏭ in the details table.
+
+Only direct dependencies gate execution. Transitive checks are gated through the chain (i.e., if A fails B, and C depends on B, C will also be skipped once B is skipped).
+
+## Examples
+
+Fail when AI (custom schema) reports an error:
+
+```yaml
+steps:
+  analyze-bug:
+    type: ai
+    schema: ./schemas/ticket-analysis.json
+    fail_if: output.error
+  log-results:
+    type: command
+    depends_on: [analyze-bug]
+    exec: echo "OK"
+```
+
+Fail when a dependency produced an error flag:
+
+```yaml
+fail_if: outputs["fetch-tickets"].error
+```
+
+Fail on text output pattern:
+
+```yaml
+steps:
+  lint:
+    type: command
+    exec: run-linter
+    fail_if: contains(output, "ERROR:")
+```
+
+## ForEach and fail_if
+
+When a check uses `forEach: true`, the engine evaluates `fail_if` once on the aggregated result (after all items complete). You can write expressions against the aggregated `output` (often an array), e.g.:
+
+```yaml
+fail_if: output.some(x => x.status === 'fail')
+```
+
+Per‑item fail logic is not evaluated via `fail_if`; use `if:` to skip item work or emit issues inside the per‑item action.
+
+## Notes
+
+- Fail conditions are evaluated during dependency‑aware execution as part of the engine (not only in providers), ensuring dependents are reliably skipped even if a provider path didn’t attach issues.
+- Issue ruleIds are consistent: `<checkName>_fail_if` for per‑check and `global_fail_if` for global.
+

package/dist/docs/failure-conditions-implementation.md

@@ -0,0 +1,271 @@
+# Enhanced Failure Condition System for Visor
+
+This document provides a complete overview of the enhanced failure condition configuration system implemented for the Visor code review tool.
+
+## Overview
+
+The enhanced failure condition system allows users to define flexible, powerful conditions using JavaScript expressions that determine when code reviews should fail or generate warnings. It supports both global conditions (applied to all checks) and check-specific conditions, with comprehensive metadata access.
+
+## Architecture
+
+### Key Components
+
+1. **FailureConditionEvaluator** (`src/failure-condition-evaluator.ts`)
+   - Core expression evaluation engine using Function Constructor with custom helper functions
+   - Processes conditions and generates structured results
+
+2. **Enhanced Configuration Types** (`src/types/config.ts`)
+   - TypeScript interfaces for failure conditions
+   - Context object definition for expression evaluation
+
+3. **CheckExecutionEngine Integration** (`src/check-execution-engine.ts`)
+   - Integration with existing check execution pipeline
+   - Automated evaluation after check completion
+
+## Configuration Schema
+
+### Simple Conditions (String Format)
+```yaml
+failure_conditions:
+  critical_blocker: "metadata.criticalIssues > 0"
+  quality_gate: "metadata.totalIssues > 10"
+```
+
+### Complex Conditions (Object Format)
+```yaml
+failure_conditions:
+  deployment_blocker:
+    condition: "metadata.criticalIssues > 0"
+    message: "Critical issues block deployment"
+    severity: error
+    halt_execution: true
+```
+
+### Check-Specific Conditions
+```yaml
+steps:
+  security:
+    type: ai
+    prompt: "Security analysis..."
+    on: [pr_opened, pr_updated]
+    failure_conditions:
+      security_gate: "metadata.errorIssues >= 1"
+```
+
+## JavaScript Expression Context
+
+The context object available to all JavaScript expressions (evaluated in a secure sandbox) includes:
+
+```typescript
+{
+  // Check results
+  issues: Issue[],           // Array of found issues
+
+  // Aggregated metadata
+  metadata: {
+    checkName: string,       // e.g., "security-check"
+    schema: string,          // e.g., "code-review", "plain"
+    group: string,           // e.g., "security-analysis"
+    totalIssues: number,
+    criticalIssues: number,
+    errorIssues: number,
+    warningIssues: number,
+    infoIssues: number
+  },
+
+  // Debug information (if available)
+  debug: {
+    errors: string[],
+    processingTime: number,
+    provider: string,
+    model: string
+  }
+}
+```
+
+## Available Helper Functions
+
+### Issue Analysis
+- `hasIssueWith(issues, field, value)` - Check if any issue has field matching value
+- `countIssues(issues, field, value)` - Count issues with field matching value
+- `hasFileWith(issues, text)` - Check if any issue file path contains text
+
+## Example Use Cases
+
+### Basic Quality Gates
+```yaml
+failure_conditions:
+  # Block on any critical issues
+  critical_blocker: "metadata.criticalIssues > 0"
+
+  # Warn on too many issues
+  quality_threshold: "metadata.totalIssues > 15"
+
+  # Schema-specific rules
+  code_review_gate: "metadata.schema == 'code-review' && metadata.errorIssues > 5"
+```
+
+### Security-Focused Conditions
+```yaml
+failure_conditions:
+  # Critical security issues
+  security_critical:
+    condition: "hasIssueWith(issues, 'category', 'security') && hasIssueWith(issues, 'severity', 'critical')"
+    message: "Critical security vulnerabilities detected"
+    severity: error
+    halt_execution: true
+
+  # Sensitive file changes
+  sensitive_files:
+    condition: "hasFileWith(issues, 'auth') || hasFileWith(issues, 'password')"
+    message: "Changes in sensitive files require extra review"
+    severity: warning
+```
+
+### Performance Monitoring
+```yaml
+failure_conditions:
+  # Slow analysis with issues
+  performance_concern:
+    condition: "debug && debug.processingTime > 30000 && metadata.errorIssues > 2"
+    message: "Analysis was slow and found multiple issues"
+    severity: info
+
+  # Database-related issues
+  database_issues:
+    condition: "hasFileWith(issues, 'database') && hasIssueWith(issues, 'category', 'performance')"
+    message: "Database performance issues detected"
+    severity: warning
+```
+
+## Implementation Details
+
+### Priority and Inheritance
+1. **Check-specific conditions** override global conditions with the same name
+2. **Global conditions** apply to all checks unless specifically overridden
+3. **Multiple conditions** are evaluated independently
+
+### Error Handling
+- Malformed JavaScript expressions result in error-level condition results
+- Missing helper functions gracefully degrade to basic JavaScript evaluation
+- All evaluation errors are captured and reported
+
+### Performance Considerations
+- JavaScript evaluation in sandbox is fast for typical expressions
+- Helper functions are optimized for common use cases
+- Condition evaluation happens after check completion (non-blocking)
+
+## Integration Points
+
+### CheckExecutionEngine
+```typescript
+// Evaluate conditions after check completion
+const results = await engine.evaluateFailureConditions(
+  checkName,
+  reviewSummary,
+  config
+);
+
+// Check if execution should halt
+if (FailureConditionEvaluator.shouldHaltExecution(results)) {
+  // Handle execution halt
+}
+```
+
+### GitHub Action Integration
+Failure conditions can be used to:
+- Set action exit codes
+- Control PR approval requirements
+- Generate status check results
+- Customize comment formatting
+
+### CLI Integration
+- Display failure condition results in CLI output
+- Use for exit code determination
+- Support debug mode for condition testing
+
+## Migration Guide
+
+### From Basic Configuration
+```yaml
+# Before
+steps:
+  security:
+    type: ai
+    prompt: "Security analysis..."
+    on: [pr_opened]
+
+# After - add failure conditions
+failure_conditions:
+  critical_gate: "metadata.criticalIssues > 0"
+
+steps:
+  security:
+    type: ai
+    prompt: "Security analysis..."
+    group: security-analysis
+    schema: code-review
+    on: [pr_opened]
+    failure_conditions:
+      security_specific: "metadata.errorIssues >= 1"
+```
+
+### Testing Conditions
+1. Use debug mode to see evaluation results
+2. Start with simple conditions and gradually add complexity
+3. Test with various issue scenarios
+4. Validate JavaScript expressions before deployment
+
+## Best Practices
+
+### Condition Design
+- Start with simple metadata-based conditions
+- Use descriptive condition names
+- Include helpful messages for complex conditions
+- Consider whether conditions should halt execution
+
+### Organization
+- Group related conditions logically
+- Use global conditions for organization-wide policies
+- Override with check-specific conditions for specialized rules
+- Document complex expressions with comments
+
+### Performance
+- Prefer metadata conditions over issue iteration
+- Use helper functions for common patterns
+- Avoid overly complex nested expressions
+- Test condition performance with large result sets
+
+## Testing
+
+### Unit Tests
+- Comprehensive coverage of FailureConditionEvaluator
+- Test various JavaScript expression patterns
+- Validate helper function behavior
+- Error handling scenarios
+
+### Integration Tests
+- End-to-end condition evaluation in CheckExecutionEngine
+- Configuration loading and validation
+- Real-world scenario testing
+- Backward compatibility verification
+
+## Future Enhancements
+
+### Potential Additions
+- Time-based conditions (e.g., business hours)
+- Environment-specific conditions (e.g., production vs staging)
+- Team-based conditions (e.g., author experience level)
+- Historical trend analysis (e.g., issue count over time)
+
+### Advanced Features
+- Condition templates and reusable expressions
+- Dynamic condition loading from external sources
+- Conditional condition evaluation based on PR metadata
+- Integration with external policy engines
+
+## Summary
+
+The enhanced failure condition system provides a flexible, powerful foundation for implementing custom quality gates and policies in Visor. By leveraging JavaScript expressions (evaluated in a secure sandbox) with comprehensive context access, teams can create sophisticated review workflows that adapt to their specific needs and standards.
+
+The system maintains full backward compatibility while opening new possibilities for automated code quality enforcement and intelligent review assistance.

package/dist/docs/failure-conditions-schema.md

@@ -0,0 +1,173 @@
+# Enhanced Failure Condition Configuration Schema
+
+This document describes the enhanced failure condition configuration system for Visor that supports JavaScript expressions (evaluated in a secure sandbox) for flexible and powerful failure evaluation.
+
+## YAML Configuration Schema
+
+### Global Failure Conditions
+
+```yaml
+version: "1.0"
+
+# Global failure conditions apply to all checks
+failure_conditions:
+  # Simple JavaScript expression for basic failure conditions
+  critical_threshold: "metadata.criticalIssues > 0"
+
+  # Complex conditions with multiple criteria
+  security_gate: "metadata.checkName == 'security' && metadata.criticalIssues > 0"
+
+  # Schema-based conditions
+  code_review_quality: "metadata.schema == 'code-review' && metadata.totalIssues > 10"
+
+  # Combined conditions with logical operators
+  deployment_blocker: "metadata.criticalIssues > 0 || (metadata.errorIssues > 5 && metadata.checkName == 'security')"
+
+# Alternative object syntax for more complex conditions
+failure_conditions:
+  critical_security_issues:
+    condition: "metadata.criticalIssues > 0 && metadata.checkName == 'security'"
+    message: "Critical security issues detected - deployment blocked"
+    severity: "error"
+
+  performance_degradation:
+    condition: "metadata.checkName == 'performance' && metadata.errorIssues >= 3"
+    message: "Performance issues detected that may impact production"
+    severity: "warning"
+
+  insufficient_coverage:
+    condition: "metadata.schema == 'code-review' && metadata.totalIssues > 15"
+    message: "Too many code quality issues - consider additional review"
+    severity: "info"
+
+steps:
+  security:
+    type: ai
+    prompt: "Analyze for security vulnerabilities..."
+    group: review
+    schema: code-review
+    on:
+      - pr_opened
+      - pr_updated
+
+    # Check-specific failure conditions override global ones
+    failure_conditions:
+      block_on_any_critical: "metadata.criticalIssues > 0"
+      warn_on_multiple_errors: "metadata.errorIssues >= 2"
+
+  performance:
+    type: ai
+    prompt: "Analyze performance implications..."
+    group: review
+    schema: code-review
+    on:
+      - pr_opened
+      - pr_updated
+
+    # Inherits global failure conditions unless overridden
+    failure_conditions:
+      performance_regression: "metadata.errorIssues > 1 && issues.some(i => i.category == 'performance')"
+```
+
+### Advanced JavaScript Expression Examples
+
+```yaml
+failure_conditions:
+  # Issue analysis with helper functions
+  sql_injection_check: "hasFileWith(issues, 'sql') && hasIssueWith(issues, 'severity', 'critical')"
+
+  # Multiple file types analysis
+  critical_files_affected: "hasFileWith(issues, '.ts') || hasFileWith(issues, '.js') && hasIssueWith(issues, 'severity', 'critical')"
+
+  # Complex metadata-based conditions
+  large_change_with_issues: "metadata.totalIssues > 5 && debug && debug.processingTime > 30000"
+
+  # Schema and group combinations
+  review_blocking_issues: "metadata.schema == 'code-review' && metadata.group == 'review' && metadata.criticalIssues > 0"
+
+  # Time-based conditions (if debug info is available)
+  slow_analysis_with_errors: "debug && debug.processingTime > 60000 && metadata.errorIssues > 0"
+
+  # File-specific security checks
+  sensitive_file_changes: "hasFileWith(issues, 'auth') || hasFileWith(issues, 'password') || hasFileWith(issues, 'secret')"
+
+  # Count-based conditions
+  multiple_security_issues: "countIssues(issues, 'category', 'security') >= 3"
+
+  # Combined conditions
+  critical_auth_issues: "hasFileWith(issues, 'auth') && countIssues(issues, 'severity', 'critical') > 0"
+```
+
+## Configuration Priority and Inheritance
+
+1. **Check-specific conditions** override global conditions with the same name
+2. **Global conditions** apply to all checks unless specifically overridden
+3. **Built-in conditions** (if any) have the lowest priority
+4. **Multiple conditions** are evaluated independently - any true condition triggers a failure
+
+## Backward Compatibility
+
+The enhanced system maintains full backward compatibility:
+
+```yaml
+# Legacy format (still supported)
+version: "1.0"
+steps:
+  security:
+    type: ai
+    prompt: "Security analysis..."
+    on:
+      - pr_opened
+      - pr_updated
+
+# Enhanced format with failure conditions
+version: "1.0"
+failure_conditions:
+  default_critical: "metadata.criticalIssues > 0"
+
+steps:
+  security:
+    type: ai
+    prompt: "Security analysis..."
+    on:
+      - pr_opened
+      - pr_updated
+
+## Interaction with Criticality
+
+Failure conditions (`fail_if`) and design‑by‑contract (`assume`, `guarantee`) work together with criticality:
+
+- Critical steps (external/control‑plane/policy):
+  - Require meaningful `assume` and `guarantee`.
+  - `continue_on_failure: false` by default; dependents skip when this step fails.
+  - Retries only for transient provider faults; no auto‑retry for logical failures (`fail_if`/`guarantee`).
+- Non‑critical steps:
+  - Contracts recommended; may allow `continue_on_failure: true`.
+  - Same retry bounds; tolerant gating.
+
+See docs/guides/fault-management-and-contracts.md for the full policy checklist and examples.
+    failure_conditions:
+      security_specific: "metadata.errorIssues >= 1"
+```
+
+## Migration Guide
+
+### Step 1: Add Global Conditions
+```yaml
+# Add to existing configuration
+failure_conditions:
+  critical_blocker: "metadata.criticalIssues > 0"
+  quality_gate: "metadata.totalIssues > 10"
+```
+
+### Step 2: Add Check-Specific Conditions
+```yaml
+steps:
+  security:
+    # existing configuration...
+    failure_conditions:
+      security_gate: "metadata.errorIssues >= 1"
+```
+
+### Step 3: Test and Refine
+Use debug mode to test conditions and refine expressions based on actual results.