@probelabs/visor 0.1.124 → 0.1.126

package/dist/docs/failure-routing-rfc.md

@@ -1,11 +1,17 @@
 # Failure Routing (Retry/Goto/Remediate) — RFC
 
-Status: Draft
+Status: **Implemented** (fully landed in engine, Phase 5 fanout/reduce included)
 
-Last updated: 2025-10-03
+Last updated: 2026-01-28
 
 Owner: Visor team
 
+> **Note**: This RFC has been fully implemented. For user documentation, see
+> [failure-routing.md](./failure-routing.md). The implementation includes all
+> core features: `on_fail`, `on_success`, `on_finish`, retry with backoff,
+> `goto`/`goto_js`, `run`/`run_js`, `goto_event`, fanout/reduce semantics, loop
+> budgets, and forEach scope isolation.
+
 ## Objectives
 
 - Enable a workflow step to handle failure by retrying, jumping back to a prior step, or running a remediation step, then continuing.
@@ -20,7 +26,9 @@ Owner: Visor team
 
 ## Config Sketch (MVP)
 
-Proposed additions use Visor’s existing 2.0 style (type/exec/depends_on). New keys are `on_fail`, `on_success`, and optional top‑level `routing` for defaults.
+Proposed additions use Visor's existing 2.0 style (type/exec/depends_on). New keys are `on_fail`, `on_success`, `on_finish`, and optional top‑level `routing` for defaults.
+
+> **Implementation note**: The implementation also added `transitions` (declarative rule-based routing) and `goto_event` (event override for goto targets). See [failure-routing.md](./failure-routing.md) for full documentation.
 
 ```yaml
 version: "2.0"
@@ -157,9 +165,10 @@ steps:
 
 ## CLI and UX
 
-- Flags: `--on-fail-max-loops`, `--retry-max`, `--no-failure-routing` (to disable feature globally).
+- **Future Flags** (not yet implemented): `--on-fail-max-loops`, `--retry-max`, `--no-failure-routing` (to disable feature globally).
 - Run summary shows failure routes taken with timestamps and attempt counts.
 - Debug: when `--debug` is set, include evaluated `*_js` results (with sensitive data redacted), sandbox timing, retry/backoff decisions, goto/run transitions, and per-scope loop counters.
+- Telemetry: routing decisions are traced via OTel events (`visor.routing`) with attributes like `trigger`, `action`, `target`, `source`, `scope`, and `goto_event`.
 
 ## Tests and Demo
 
@@ -168,14 +177,24 @@ steps:
 
 ## Acceptance Criteria
 
-- Goto: Given a failing step with `goto: setup-env`, engine jumps to `setup-env`, proceeds, and re-runs the failed step.
-- Remediation: Given `run: [lint-fix]`, if remediation succeeds, the failed step re-runs once; if remediation fails, the run stops with a clear message.
-- Retry: Per-step retries respect backoff and caps; global `max_loops` prevents infinite ping-pong.
-- Compatibility: Configs without `on_fail` behave exactly as today.
-- Observability: Logs show ordered trace of retries and jumps; exit codes reflect final outcome.
-- Dynamic routing: `goto_js` and `run_js` work with pure, time-limited evaluation; precedence and merging behave as specified.
-- forEach: Each item runs with isolated counters; `*_js` receives `foreach` context and cannot jump across scopes.
-- on_success goto: After a step succeeds, `goto` (ancestor-only) can jump back to a prior step; with `max_loops` enforcement the run either converges or fails with a clear trace.
+All original acceptance criteria have been met:
+
+- [x] **Goto**: Given a failing step with `goto: setup-env`, engine jumps to `setup-env`, proceeds, and re-runs the failed step.
+- [x] **Remediation**: Given `run: [lint-fix]`, if remediation succeeds, the failed step re-runs once; if remediation fails, the run stops with a clear message.
+- [x] **Retry**: Per-step retries respect backoff and caps; global `max_loops` prevents infinite ping-pong.
+- [x] **Compatibility**: Configs without `on_fail` behave exactly as today.
+- [x] **Observability**: Logs show ordered trace of retries and jumps; exit codes reflect final outcome.
+- [x] **Dynamic routing**: `goto_js` and `run_js` work with pure, time-limited evaluation; precedence and merging behave as specified.
+- [x] **forEach**: Each item runs with isolated counters; `*_js` receives `foreach` context and cannot jump across scopes.
+- [x] **on_success goto**: After a step succeeds, `goto` (ancestor-only) can jump back to a prior step; with `max_loops` enforcement the run either converges or fails with a clear trace.
+
+Additional features implemented beyond the original RFC:
+
+- [x] **on_finish hook**: Runs after all forEach iterations and dependent checks complete — ideal for aggregation.
+- [x] **goto_event**: Override the event trigger when performing a goto (e.g., simulate `pr_updated`).
+- [x] **Declarative transitions**: Rule-based routing via `transitions: [{ when, to, goto_event }]` in on_fail/on_success/on_finish.
+- [x] **Fanout/reduce**: Control whether routing targets run per-item (`fanout: map`) or as a single aggregation (`fanout: reduce`).
+- [x] **Criticality-aware retry suppression**: High-criticality steps skip retries for logical failures (fail_if/guarantee).
 
 ## Open Questions
 
@@ -184,10 +203,16 @@ steps:
 - Any preferred global default (e.g., retry once everywhere with linear 2s backoff)?
 - Should we allow opt-in cross-scope targets via explicit qualifiers (e.g., `parent:setup-env`), guarded by additional loop caps?
 
-## Next Steps
+## Next Steps (Completed)
+
+All primary implementation work is done:
+
+1. ~~Audit current workflow engine & failure handling.~~ Done.
+2. ~~Finalize config keys and schema validation messages.~~ Done — see `src/types/config.ts`.
+3. ~~Implement engine changes with loop safeguards.~~ Done — see `src/state-machine/states/routing.ts`.
+4. ~~Add tests and demos.~~ Done — see `tests/integration/routing-*.test.ts` and `examples/routing-*.yaml`.
+5. ~~Extend docs and workshop slides.~~ Done — see [failure-routing.md](./failure-routing.md).
 
-1. Audit current workflow engine & failure handling.
-2. Finalize config keys and schema validation messages.
-3. Implement engine changes with loop safeguards.
-4. Add tests and the `lab-05-retry.yaml` demo.
-5. Extend docs and workshop slides.
+Remaining follow-ups:
+- CLI flags (`--on-fail-max-loops`, `--retry-max`, `--no-failure-routing`) are planned but not yet implemented.
+- Consider adding labeled checkpoints and opt-in cross-scope targets (see Open Questions).

package/dist/docs/failure-routing.md

@@ -77,7 +77,8 @@ steps:
     on_success:
       run: [notify]
       goto_js: |
-        return attempt === 1 ? 'unit' : null;  # only once
+        // Jump back only once using history length as proxy for attempt count
+        return outputs.history['build'].length === 1 ? 'unit' : null;
   notify: { type: command, exec: "echo notify" }
 ```
 
@@ -117,10 +118,45 @@ Per-step actions:
   - `retry`: `{ max, backoff: { mode: fixed|exponential, delay_ms } }`
   - `run`: `[step-id, …]`
   - `goto`: `step-id` (ancestor-only)
+  - `goto_event`: event to simulate during goto (e.g., `pr_updated`)
   - `run_js`: JS returning `string[]`
   - `goto_js`: JS returning `string | null`
+  - `transitions`: declarative transition rules (see below)
 - `on_success`:
-  - `run`, `goto`, `run_js`, `goto_js` (same types and constraints as above)
+  - `run`, `goto`, `goto_event`, `run_js`, `goto_js`, `transitions` (same types and constraints as above)
+- `on_finish` (forEach checks only):
+  - `run`, `goto`, `goto_event`, `run_js`, `goto_js`, `transitions` (same types and constraints as above)
+
+### Declarative Transitions
+
+The `transitions` field provides a declarative alternative to `goto_js`. It is an array of rules evaluated in order; the first matching rule wins:
+
+```yaml
+steps:
+  validate:
+    type: ai
+    on_success:
+      transitions:
+        - when: "outputs['validate'].score >= 90"
+          to: publish
+        - when: "outputs['validate'].score >= 70"
+          to: review
+        - when: "true"  # default fallback
+          to: reject
+```
+
+Each rule has:
+- `when`: JavaScript expression evaluated in the same sandbox as `goto_js`
+- `to`: target step ID, or `null` to explicitly skip goto
+- `goto_event`: optional event override (same as `goto_event` above)
+
+Helper functions available in `when` expressions:
+- `any(arr, pred)` - returns true if any element matches predicate
+- `all(arr, pred)` - returns true if all elements match predicate
+- `none(arr, pred)` - returns true if no element matches predicate
+- `count(arr, pred)` - returns count of elements matching predicate
+
+When `transitions` is present, it takes precedence over `goto_js` and `goto`.
 
 ## Semantics
 
@@ -128,7 +164,7 @@ Per-step actions:
 - Run: on failure (or success), run listed steps first; if successful, the failed step is re-attempted once (failure path).
 - Goto (ancestor-only): jump back to a previously executed dependency, then continue forward. On success, Visor re-runs the current step once after the jump.
 - Loop safety: `routing.max_loops` counts all routing transitions (runs, gotos, retries). Exceeding it aborts the current scope with a clear error. For a hard cap on repeated executions of the same step, see [Execution Limits](./limits.md).
-- forEach: each item is isolated with its own loop/attempt counters; `*_js` receives `{ foreach: { index, total, parent } }`.
+- forEach: each item is isolated with its own loop/attempt counters; `*_js` in on_finish context receives forEach metadata (see [Available Context in on_finish](#available-context-in-on_finish)).
 
 ### Fan‑out vs. Reduce (Phase 5)
 
@@ -214,7 +250,8 @@ steps:
     on: [pr_opened, pr_updated]
     on_success:
       goto_js: |
-        return attempt === 1 ? 'overview' : null
+        // Jump back only once using history length as proxy for attempt count
+        return outputs.history['quality'].length === 1 ? 'overview' : null
       goto_event: pr_updated
 ```
 
@@ -225,20 +262,26 @@ When to use goto_event vs. full re-run:
 ## Dynamic JS (safe, sync only)
 
 - `goto_js` / `run_js` are evaluated in a sandbox with:
-  - Read-only context: `{ step, attempt, loop, error, foreach, outputs, pr, files, env }`
+  - Read-only context: `{ step, outputs, outputs_history, outputs_raw, output, memory, event, forEach }`
   - `outputs` contains current values, `outputs.history` contains arrays of all previous values (see [Output History](./output-history.md))
+  - `outputs_raw` provides aggregate/parent values (e.g., full array from forEach parent)
+  - `memory` provides read/write access to the memory store (get, set, has, getAll, increment, clear)
+  - `log()` function available for debugging (outputs with `Debug:` prefix)
   - Pure sync execution; no IO, no async, no timers, no require/process.
   - Time and size limits (short wall time; small code/output caps) — evaluation failures fall back to static routing.
 
+Note: The `on_finish` context is richer and additionally includes `attempt`, `loop`, `pr`, `files`, and `env`.
+
 Return types:
 - `goto_js`: a `string` (step id) or `null`/`undefined` to skip.
 - `run_js`: a `string[]` (may be empty). Duplicates are removed preserving order.
 
 ## Guardrails & Tips
 
-- Keep `max_loops` small (5–10). Add retries sparingly and prefer remediation over blind loops.
+- Keep `max_loops` small (5-10). Add retries sparingly and prefer remediation over blind loops.
 - Restrict `goto` to ancestors to preserve dependency semantics and avoid hard-to-reason paths.
-- For expensive remediations, put them behind `run_js` conditions keyed to `error.message`/`stderr`.
+- For expensive remediations, put them behind `run_js` conditions keyed to `output` or `outputs.history` values.
+- Use `outputs.history['check-name'].length` as a proxy for attempt count in `goto_js`/`run_js`.
 - In CI, you can override defaults with CLI flags (future): `--on-fail-max-loops`, `--retry-max`.
 
 ## on_finish Hook (forEach Aggregation & Routing)
@@ -289,30 +332,44 @@ The `on_finish` hooks have access to the complete execution context:
 {
   step: { id: 'extract-facts', tags: [...], group: '...' },
   attempt: 1,              // Current attempt number for this check
-  loop: 2,                 // Current loop number in routing
+  loop: 0,                 // Current loop number in routing
   outputs: {
-    'extract-facts': [...], // Array of forEach items
-    'validate-fact': [...], // Array of ALL dependent results
+    'extract-facts': [...],   // Array of forEach items
+    'validate-fact': [...],   // Array of ALL dependent results
+    history: { ... }          // Alias for outputs_history
   },
-  outputs.history: {
+  outputs_history: {
     'extract-facts': [[...], ...], // Cross-loop history
     'validate-fact': [[...], ...], // All results from all iterations
   },
-  // Alias (also available):
-  outputs_history: {
-    'extract-facts': [[...], ...],
-    'validate-fact': [[...], ...],
+  outputs_raw: {
+    'extract-facts': [...],   // Aggregate/parent values
+    'validate-fact': [...],
   },
   forEach: {
-    total: 3,              // Total number of items
-    successful: 3,         // Number of successful iterations
-    failed: 0,             // Number of failed iterations
-    items: [...]           // The forEach items array
+    items: 3,              // Number of forEach items
+    last_wave_size: 3,     // Items in the last wave (when forEach parent)
+    last_items: [...],     // The forEach items array (when forEach parent)
+    is_parent: true        // Indicates this check is a forEach parent
+  },
+  memory: {                // Memory access functions
+    get: (key, ns?) => ...,
+    set: (key, value, ns?) => ...,
+    has: (key, ns?) => ...,
+    getAll: (ns?) => ...,
+    increment: (key, amount?, ns?) => ...,
+    clear: (ns?) => ...
+  },
+  pr: {                    // PR metadata
+    number: 123,
+    title: '...',
+    author: '...',
+    branch: '...',
+    base: '...'
   },
-  memory,                  // Memory access functions
-  pr,                      // PR metadata
-  files,                   // Changed files
-  env                      // Environment variables
+  files: [...],            // Changed files
+  env: { ... },            // Environment variables (filtered for safety)
+  event: { name: '...' }   // Event that triggered execution
 }
 ```