@probelabs/visor 0.1.106 → 0.1.111

package/dist/docs/schema-templates.md

@@ -0,0 +1,68 @@
+## 📋 Schema-Template System
+
+Visor pairs JSON Schemas (data shape) with Liquid templates (rendering) so outputs are predictable, auditable, and GitHub‑native.
+
+### Overview
+- Schema validates check output at runtime (via AJV)
+- Template renders tables/markdown and GitHub Checks annotations
+- Group controls which GitHub comment a check posts to
+
+### Quick Example
+
+```yaml
+steps:
+  security:
+    type: ai
+    group: code-review
+    schema: code-review
+    prompt: "Review for security issues and return JSON"
+
+  overview:
+    type: ai
+    group: summary
+    schema: text
+    prompt: "Summarize PR in markdown"
+```
+
+### Built-in Schemas
+- code-review: structured findings with severity, file, line → native annotations
+- text: free‑form markdown content (no annotations)
+
+### Grouping
+
+```yaml
+steps:
+  security:   { group: code-review }
+  performance:{ group: code-review }
+  overview:   { group: summary }
+  assistant:  { group: dynamic } # always creates a new comment
+```
+
+### Custom Schemas
+
+```yaml
+schemas:
+  custom-metrics:
+    file: ./schemas/metrics.json
+
+steps:
+  metrics:
+    schema: custom-metrics
+    group: metrics
+```
+
+### GitHub Checks API Compatibility
+
+For status checks and annotations, use structured output with `issues[]` having:
+- severity: critical | error | warning | info
+- file, line, message
+
+Unstructured (none/plain) → posted as-is, no status checks.
+
+### Enhanced Prompts
+- Smart auto‑detection, Liquid templating, file‑based prompts
+- Template context: `pr`, `files`, `event`, `outputs`, `utils`
+- See [Liquid Templates Guide](./liquid-templates.md) for available variables and filters
+
+See full examples in `defaults/.visor.yaml`.
+

package/dist/docs/script.md

@@ -0,0 +1,34 @@
+## Script step (type: `script`)
+
+The `script` provider executes JavaScript in a secure sandbox with access to
+PR context, dependency outputs, and the Visor memory store.
+
+- Use `type: script` with a `content` block containing your code.
+- The sandbox exposes these objects:
+  - `pr`: basic PR metadata and file list.
+  - `outputs`: map of dependency outputs (plus `outputs.history`).
+  - `outputs_raw`: aggregated values from `-raw` dependencies.
+  - `outputs_history_stage`: per-stage output history slice for tests.
+  - `memory`: synchronous helpers `get`, `set`, `append`, `increment`, `delete`, `clear`.
+- The value you `return` becomes this step’s `output` (for `depends_on`).
+
+Example:
+
+```yaml
+steps:
+  extract-facts:
+    type: command
+    exec: node ./scripts/extract-facts.js
+
+  aggregate:
+    type: script
+    depends_on: [extract-facts]
+    content: |
+      const facts = outputs['extract-facts'] || [];
+      memory.set('total_facts', Array.isArray(facts) ? facts.length : 0, 'fact-validation');
+      const allValid = Array.isArray(facts) && facts.every(f => f.valid === true);
+      memory.set('all_valid', allValid, 'fact-validation');
+      return { total: memory.get('total_facts', 'fact-validation'), allValid };
+```
+
+The script context and memory helpers mirror other providers’ contexts.

package/dist/docs/sdk.md

@@ -0,0 +1,222 @@
+# Visor SDK (Programmatic Usage)
+
+Run Visor from Node.js without shelling out. The SDK is a thin façade over the existing engine: it just wires inputs/outputs and reuses all core behavior (routing, providers, templates, etc.).
+
+## Install
+
+```bash
+npm i -D @probelabs/visor
+```
+
+## Quick Start
+
+### JavaScript (ESM)
+```ts
+import { loadConfig, runChecks } from '@probelabs/visor/sdk';
+
+// Load config from object (not file!) - validation and defaults applied
+const config = await loadConfig({
+  version: '1.0',
+  checks: {
+    'security': { type: 'command', exec: 'npm audit' },
+    'lint': { type: 'command', exec: 'npm run lint' },
+  }
+});
+
+const result = await runChecks({
+  config,
+  checks: Object.keys(config.checks),
+  output: { format: 'json' },
+});
+console.log('Total issues:', result.reviewSummary.issues?.length ?? 0);
+```
+
+### TypeScript
+```typescript
+import { loadConfig, runChecks, type VisorConfig, type RunOptions } from '@probelabs/visor/sdk';
+
+// Type-safe config construction
+const rawConfig: Partial<VisorConfig> = {
+  version: '1.0',
+  checks: {
+    'security': { type: 'command', exec: 'npm audit' },
+    'lint': { type: 'command', exec: 'npm run lint' },
+  }
+};
+
+const config = await loadConfig(rawConfig);
+const result = await runChecks({
+  config,
+  checks: Object.keys(config.checks),
+  output: { format: 'json' },
+});
+
+// Type-safe result access with full type inference
+console.log('Total issues:', result.reviewSummary.issues?.length ?? 0);
+console.log('Checks executed:', result.checksExecuted);
+console.log('Execution time:', result.executionTime);
+```
+
+CommonJS
+```js
+const { loadConfig, runChecks } = require('@probelabs/visor/sdk');
+(async () => {
+  const config = await loadConfig({
+    version: '1.0',
+    checks: { test: { type: 'command', exec: 'echo test' } }
+  });
+  const result = await runChecks({ config, checks: Object.keys(config.checks), output: { format: 'json' } });
+  console.log('Total issues:', result.reviewSummary.issues?.length ?? 0);
+})();
+```
+
+### Loading from Files
+
+You can also load config from files:
+
+```ts
+import { loadConfig, runChecks } from '@probelabs/visor/sdk';
+
+// Load from specific file path
+const config = await loadConfig('./my-config.yaml');
+
+// Or discover default (.visor.yaml/.visor.yml)
+const config2 = await loadConfig();
+
+const result = await runChecks({
+  config,
+  checks: Object.keys(config.checks),
+  output: { format: 'json' },
+});
+```
+
+**Note:** The `output` parameter in `runChecks()` options controls the CLI output format (table/json/markdown/sarif). The config's `output` field is for GitHub PR comments and is optional for programmatic use.
+
+### Strict Validation Mode
+
+By default, unknown config keys generate warnings but don't fail. Enable strict mode to catch config errors early:
+
+```ts
+import { runChecks } from '@probelabs/visor/sdk';
+
+const config = {
+  version: '1.0',
+  checks: { test: { type: 'command', exec: 'echo test' } },
+  typo_field: 'oops'  // This would normally just warn
+};
+
+try {
+  await runChecks({
+    config,
+    checks: ['test'],
+    strictValidation: true  // Now throws error for unknown keys
+  });
+} catch (error) {
+  console.error('Config error:', error.message);
+  // Error: Unknown top-level key 'typo_field' will be ignored.
+}
+```
+
+## API
+
+- `loadConfig(configOrPath?: string | Partial<VisorConfig>, options?: { strict?: boolean }): Promise<VisorConfig>`
+  - Loads and validates a config from an object, file path, or discovers defaults
+  - Accepts config objects (validates and applies defaults) or file paths
+  - Returns fully validated config with all defaults applied
+  - Set `options.strict` to treat warnings as errors
+- `resolveChecks(checkIds: string[], config: VisorConfig | undefined): string[]`
+  - Expands check IDs to include dependencies in the correct order.
+- `runChecks(options: RunOptions): Promise<AnalysisResult>`
+  - Runs checks programmatically. Thin wrapper around the engine's `executeChecks`.
+
+### Types
+
+- `RunOptions`
+  - `config?: VisorConfig` | `configPath?: string`
+  - `checks?: string[]`
+  - `cwd?: string`
+  - `timeoutMs?: number`
+  - `output?: { format?: 'table'|'json'|'markdown'|'sarif' }`
+  - `debug?: boolean`
+  - `maxParallelism?: number`
+  - `failFast?: boolean`
+  - `tagFilter?: { include?: string[]; exclude?: string[] }`
+  - `strictValidation?: boolean` - Treat config warnings (unknown keys) as errors (default: false)
+
+- `AnalysisResult`
+  - `reviewSummary.issues: Issue[]`
+  - `executionTime: number`, `timestamp: string`, `checksExecuted: string[]`
+
+Refer to `src/types/config.ts` for `VisorConfig`, `Issue`, and related types.
+
+## Safety & Criticality (Quick Note)
+
+When building configs programmatically, model safety explicitly:
+
+- Declare criticality on steps with `criticality: external|internal|policy|info`.
+- Add contracts to critical steps:
+  - `assume:` preconditions checked before execution
+  - `guarantee:` postconditions checked after execution
+- Use declarative `transitions` for routing rather than `goto_js`.
+
+Example config (JS object):
+```ts
+const cfg = await loadConfig({
+  version: '1.0',
+  checks: {
+    'post-comment': {
+      type: 'github',
+      criticality: 'external',
+      on: ['pr_opened'],
+      op: 'comment.create',
+      assume: ["isMember()"],
+      guarantee: ["output && typeof output.id === 'number'"],
+      continue_on_failure: false,
+    },
+    // Structured outputs with unified `schema` (object) for validation
+    'summarize-json': {
+      type: 'ai',
+      schema: {
+        type: 'object',
+        properties: { ok: { type: 'boolean' }, items: { type: 'array', items: { type: 'string' } } },
+        required: ['ok', 'items']
+      },
+      prompt: 'Return JSON with ok and items...'
+    },
+    // Command/script can also use JSON Schema via `schema`
+    'aggregate': {
+      type: 'script',
+      content: 'return { all_valid: true };',
+      schema: { type: 'object', properties: { all_valid: { type: 'boolean' } }, required: ['all_valid'], additionalProperties: false }
+    }
+  },
+});
+```
+
+## Notes
+
+- SDK adds no new sandboxing or providers; all safety lives in the core engine.
+- For offline demos, unset provider env vars if you rely on mock providers.
+- You can still use all CLI features alongside the SDK in the same project.
+
+## Examples (in repo)
+
+- `examples/sdk-basic.mjs` (ESM) – Minimal example with raw config object
+- `examples/sdk-cjs.cjs` (CJS) – CommonJS usage
+- `examples/sdk-manual-config.mjs` (ESM) – Manual config construction with validation
+- `examples/sdk-typescript.ts` (TypeScript) – **Type-safe example** showing:
+  - Full TypeScript type safety with SDK type definitions
+  - Importing types from `@probelabs/visor/sdk`
+  - Type inference for configs and results
+  - Compile-time type checking
+  - Using exported types (`VisorConfig`, `RunOptions`)
+  - To run: `npx tsc examples/sdk-typescript.ts --module esnext --target es2022 --moduleResolution bundler --esModuleInterop --skipLibCheck && node examples/sdk-typescript.js`
+- `examples/sdk-comprehensive.mjs` (ESM) – **Complex example** showing:
+  - Multi-level check dependencies (`depends_on`)
+  - Tag filtering
+  - Parallel execution control
+  - Dependency resolution with `resolveChecks()`
+  - Strict validation mode
+  - Complete pipeline execution
+
+These are also exercised by CI smoke tests.

package/dist/docs/security.md

@@ -0,0 +1,7 @@
+## 🔐 Security Defaults
+
+- Prefer GitHub App for bot identity and granular org/repo permissions.
+- Lock remote extends with `--allowed-remote-patterns`; default is off.
+- No external network calls unless configured (providers, HTTP checks, etc.).
+- Document where tokens are used; rotate provider keys regularly.
+

package/dist/docs/suppressions.md

@@ -0,0 +1,89 @@
+## 🔇 Suppressing Warnings
+
+Visor supports suppressing specific warnings or all warnings in a file using special comments in your code. This is useful for false positives or intentional code patterns that should not trigger warnings.
+
+### Line-Level Suppression
+
+Add `visor-disable` in a comment within ±2 lines of the issue to suppress it:
+
+```javascript
+// Example: Suppress a specific warning
+function authenticate() {
+  const testPassword = "demo123"; // visor-disable
+  // This hardcoded password warning will be suppressed
+}
+```
+
+The suppression works with any comment style:
+- `// visor-disable` (JavaScript, TypeScript, C++, etc.)
+- `# visor-disable` (Python, Ruby, Shell, etc.)
+- `/* visor-disable */` (Multi-line comments)
+- `<!-- visor-disable -->` (HTML, XML)
+
+### File-Level Suppression
+
+To suppress all warnings in an entire file, add `visor-disable-file` in the first 5 lines:
+
+```javascript
+// visor-disable-file
+// All warnings in this file will be suppressed
+
+function insecureCode() {
+  eval("user input"); // No warning
+  const password = "hardcoded"; // No warning
+}
+```
+
+### Configuration
+
+The suppression feature is enabled by default. You can disable it in your configuration:
+
+```yaml
+# .visor.yaml
+version: "1.0"
+output:
+  suppressionEnabled: false  # Disable suppression comments
+  pr_comment:
+    format: markdown
+    group_by: check
+```
+
+### Important Notes
+
+- Suppression comments are case-insensitive (`visor-disable`, `VISOR-DISABLE`, `Visor-Disable`)
+- The comment just needs to contain the suppression keyword as a substring
+- When issues are suppressed, Visor logs a summary showing which files had suppressed issues
+- Use suppression judiciously - it's better to fix issues than suppress them
+
+### Examples
+
+```python
+# Python example
+def process_data():
+    api_key = "sk-12345"  # visor-disable
+    return api_key
+```
+
+```typescript
+// TypeScript example - suppress within range
+function riskyOperation() {
+  // visor-disable
+  const unsafe = eval(userInput); // Suppressed (within 2 lines)
+  processData(unsafe);             // Suppressed (within 2 lines)
+
+  doSomethingElse();
+  anotherOperation();              // NOT suppressed (> 2 lines away)
+}
+```
+
+```go
+// Go example - file-level suppression
+// visor-disable-file
+package main
+
+func main() {
+    password := "hardcoded" // All issues suppressed
+    fmt.Println(password)
+}
+```
+

package/dist/docs/tag-filtering.md

@@ -0,0 +1,258 @@
+## 🏷️ Tag-Based Check Filtering
+
+Visor supports tagging checks to create flexible execution profiles. This lets you run different sets of checks in different environments (e.g., lightweight checks locally, comprehensive checks in CI).
+
+### How It Works
+
+1. Tag your checks with descriptive labels
+2. Filter execution using `--tags` and `--exclude-tags`
+3. Dependencies adapt intelligently based on what’s included
+
+Note on defaults
+- If you do NOT provide any tag filter (no `--tags`/`--exclude-tags` and no `tag_filter` in config), Visor only runs untagged checks. Any check that has `tags: [...]` is skipped by default. This keeps day‑to‑day runs lightweight and makes tagged groups opt‑in.
+- To run tagged checks, explicitly include their tags (for example, `--tags github,security`).
+
+### Basic Configuration
+
+```yaml
+# .visor.yaml
+version: "1.0"
+
+steps:
+  # Fast, local security check
+  security-quick:
+    type: ai
+    prompt: "Quick security scan for common vulnerabilities"
+    tags: ["local", "fast", "security"]
+    on: [pr_opened, pr_updated]
+
+  # Comprehensive security analysis (for CI)
+  security-comprehensive:
+    type: ai
+    prompt: "Deep security analysis with full vulnerability scanning"
+    tags: ["remote", "comprehensive", "security", "slow"]
+    on: [pr_opened]
+
+  # Performance check that runs everywhere
+  performance:
+    type: ai
+    prompt: "Analyze performance issues"
+    tags: ["local", "remote", "performance", "fast"]
+    on: [pr_opened, pr_updated]
+
+  # Experimental new check
+  ai-architecture:
+    type: ai
+    prompt: "AI-powered architecture review"
+    tags: ["experimental", "architecture", "slow"]
+    on: [manual]
+
+  # Report that depends on security checks
+  security-report:
+    type: noop
+    tags: ["reporting", "local", "remote"]
+    depends_on: [security-quick, security-comprehensive]
+    on: [pr_opened, pr_updated]
+```
+
+### CLI Usage
+
+```bash
+# Run only fast, local checks (great for pre-commit hooks)
+visor --tags local,fast
+
+# Run comprehensive remote checks (for CI/CD)
+visor --tags remote,comprehensive
+
+# Run all security-related checks
+visor --tags security
+
+# Run everything except slow checks
+visor --exclude-tags slow
+
+# Run everything except experimental features
+visor --exclude-tags experimental
+
+# Combine filters: Run fast security checks only
+visor --tags security,fast
+
+# Run local checks but skip experimental ones
+visor --tags local --exclude-tags experimental
+```
+
+### GitHub Action Usage
+
+```yaml
+name: Code Review with Tags
+on: pull_request
+
+jobs:
+  # Fast checks on every push
+  fast-review:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: probelabs/visor@v1
+        with:
+          tags: "local,fast"
+          exclude-tags: "experimental"
+        env:
+          GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+
+  # Comprehensive checks only on main branch PRs
+  comprehensive-review:
+    if: github.base_ref == 'main'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: probelabs/visor@v1
+        with:
+          tags: "remote,comprehensive"
+        env:
+          GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+```
+
+### Common Tag Patterns
+
+| Tag | Purpose | Example Use |
+|-----|---------|-------------|
+| `local` | Checks suitable for local development | Pre-commit hooks, developer testing |
+| `remote` | Checks designed for CI/CD environments | GitHub Actions, Jenkins |
+| `fast` | Quick checks (< 30 seconds) | Rapid feedback loops |
+| `slow` | Time-consuming checks | Nightly builds, release validation |
+| `security` | Security-related checks | Security audits |
+| `performance` | Performance analysis | Performance testing |
+| `style` | Code style and formatting | Linting, formatting |
+| `experimental` | Beta/testing features | Opt-in testing |
+| `critical` | Must-pass checks | Release gates |
+| `comprehensive` | Thorough analysis | Full PR reviews |
+
+### Advanced Examples
+
+#### Default behavior vs. explicit tags
+
+By default (no tag filter provided), only untagged checks execute. To include tagged checks, specify them explicitly:
+
+```bash
+# Default (no flags): only untagged checks
+visor
+
+# Include github-tagged checks (e.g., GitHub operations)
+visor --tags github
+
+# Include multiple tag groups
+visor --tags github,security
+```
+
+In the test runner, you can mirror this behavior with the tests DSL:
+
+```yaml
+# defaults/.visor.tests.yaml
+tests:
+  defaults:
+    # Run GitHub-tagged checks during tests
+    tags: "github"
+```
+
+#### Environment-Specific Execution
+
+```yaml
+# Development environment - fast feedback
+development:
+  extends: .visor.yaml
+  tag_filter:
+    include: ["local", "fast"]
+    exclude: ["experimental"]
+
+# Staging environment - balanced
+staging:
+  extends: .visor.yaml
+  tag_filter:
+    include: ["remote", "security", "performance"]
+    exclude: ["experimental"]
+
+# Production environment - comprehensive
+production:
+  extends: .visor.yaml
+  tag_filter:
+    include: ["remote", "comprehensive", "critical"]
+```
+
+#### Multi-Stage Pipeline
+
+```yaml
+# GitHub Actions workflow with progressive checks
+name: Progressive Code Review
+on: pull_request
+
+jobs:
+  stage-1-fast:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: probelabs/visor@v1
+        with:
+          tags: "fast,critical"
+          fail-fast: "true"  # Stop if critical issues found
+
+  stage-2-security:
+    needs: stage-1-fast
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: probelabs/visor@v1
+        with:
+          tags: "security"
+          exclude-tags: "fast"  # Run deeper security checks
+
+  stage-3-comprehensive:
+    needs: [stage-1-fast, stage-2-security]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: probelabs/visor@v1
+        with:
+          tags: "comprehensive"
+          exclude-tags: "fast,security"  # Run remaining checks
+```
+
+#### Dependency-Aware Filtering
+
+When using tags with dependencies, Visor intelligently handles missing dependencies:
+
+```yaml
+steps:
+  data-validation:
+    type: ai
+    prompt: "Validate data structures"
+    tags: ["local", "data"]
+
+  api-validation:
+    type: ai
+    prompt: "Validate API contracts"
+    tags: ["remote", "api"]
+
+  integration-report:
+    type: noop
+    tags: ["reporting"]
+    depends_on: [data-validation, api-validation]
+    # When filtered by "local" tag, only uses data-validation
+    # When filtered by "remote" tag, only uses api-validation
+    # With no filter, uses both dependencies
+```
+
+### Tag Validation Rules
+
+- Tags must start with an alphanumeric character
+- Can contain letters, numbers, hyphens, and underscores
+- Examples: `local`, `test-env`, `feature_flag`, `v2`
+- Invalid: `-invalid`, `@special`, `tag with spaces`
+
+### Best Practices
+
+1. Use consistent naming conventions across your organization
+2. Document your tag taxonomy in your team's wiki
+3. Start simple: begin with `local`/`remote` or `fast`/`slow`
+4. Avoid over-tagging to reduce confusion
+5. Use tag combinations for fine-grained control
+6. Test tag filters before deploying broadly