loreli 0.0.0 → 1.0.0

package/packages/risk/README.md

@@ -0,0 +1,119 @@
+# loreli/risk
+
+Risk assessment workflow for Loreli's orchestration pipeline. Extends the `Workflow` base class to manage risk agents — dispatching them for new PRs, reading their verdicts, and applying risk labels that gate reviewer dispatch in `loreli/review`.
+
+## Research Findings
+
+No existing npm packages perform LLM-driven PR risk assessment with label-based routing. This is domain-specific to Loreli's adversarial review model.
+
+## How It Works
+
+RiskWorkflow runs **before** ReviewWorkflow in the reactor chain. When a new PR appears from an action agent, the risk workflow:
+
+1. **Assess** — Dispatches an opposing-provider risk agent with the PR's diff, file stats, linked issue body, and planning objective.
+2. **Verdict** — Reads the risk agent's verdict comment (posted via the `comment` MCP tool with `risk: true`). The comment tool applies the appropriate GitHub label as a side effect.
+3. **Route** — Based on the verdict label:
+   - `loreli:low-risk` — PR passes through to `ReviewWorkflow.scan()` for normal reviewer dispatch.
+   - `loreli:medium-risk` — PR passes through with risk context attached to the reviewer prompt.
+   - `loreli:critical-risk` — PR is escalated to HITL. No reviewer is dispatched.
+
+The contract between risk and review is **GitHub labels** — visible, auditable, and filterable. No shared in-memory state is needed.
+
+## API Reference
+
+### `RiskWorkflow` (extends Workflow)
+
+```js
+import { RiskWorkflow } from 'loreli/risk';
+
+const risk = new RiskWorkflow(orchestrator, hub);
+```
+
+#### Static Properties
+
+| Property | Value | Description |
+|----------|-------|-------------|
+| `role` | `'risk'` | Agent role this workflow manages |
+| `template` | `prompts/risk.md` | Mustache template for risk assessment prompts |
+
+### Methods
+
+#### `risk.assess(repo)` → Promise\<Array\<{pr, agent}\>\>
+
+Scan for new PRs from action agents and dispatch risk agents. Skips PRs that:
+- Already have a risk label (previously assessed)
+- Are currently being assessed (`_assessing` map)
+- Were already assessed (`_assessed` set)
+
+Returns early with an empty array when `review.skipRiskAssessment` is `true`.
+
+When enlisting a risk agent fails, the PR is marked as assessed so `ReviewWorkflow.scan()` can proceed without the risk gate blocking it.
+
+#### `risk.verdict(repo)` → Promise\<Array\<{pr, level}\>\>
+
+Check for risk verdicts on PRs being assessed. For each PR in `_assessing`:
+- Looks for a comment with the `risk` marker
+- Parses the `level` field (`LOW`, `MEDIUM`, `CRITICAL`)
+- Kills the risk agent
+- Moves the PR from `_assessing` to `_assessed`
+
+If the stall timeout is exceeded without a verdict, the PR is marked as assessed with level `TIMEOUT`.
+
+#### `risk.reap(repo)` → Promise\<void\>
+
+Clean up stale risk assessments. Risk agents that die without posting a verdict leave PRs stuck in `_assessing`. This handler checks for dead agents (no longer in the orchestrator's agents map), marks their PRs as assessed, and applies a fallback `loreli:low-risk` label so `ReviewWorkflow.scan()` can proceed. Without the fallback label, the PR would be assessed in memory but unlabeled on GitHub — permanently skipped by the review label gate.
+
+#### `risk.closesIssue(body)` → number|null
+
+Extract the first "Closes #N" or "Fixes #N" issue number from a PR body.
+
+#### `risk.objective(repo, childNumber)` → Promise\<string\>
+
+Resolve the planning objective from the parent issue by finding the parent via `loreli:parent` label and sub-issue linkage.
+
+### Reactor Handlers
+
+```js
+risk.reactor()
+// → { 'risk-hydrate': fn, assess: fn, verdict: fn, 'risk-reap': fn }
+```
+
+These must be registered **before** review handlers in the orchestrator so labels are applied before `scan()` checks for them.
+
+### Internal State
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `_assessing` | `Map<number, {riskAgent, dispatchedAt}>` | PRs currently awaiting a risk verdict |
+| `_assessed` | `Set<number>` | PRs that have completed risk assessment |
+
+## Configuration
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `review.skipRiskAssessment` | `boolean` | `false` | When `true`, disables the risk workflow entirely. `assess()` returns early and `ReviewWorkflow.scan()` skips the label gate. |
+
+## Prompt Template
+
+The risk prompt (`prompts/risk.md`) provides the risk agent with:
+- The original planning objective
+- The linked issue body
+- PR metadata (number, title, branch, author)
+- File change stats (filename, status, additions, deletions)
+- Unified diff
+
+The agent posts its verdict using the `comment` MCP tool with `risk: true` and the appropriate `level`. The comment tool applies the `loreli:{level}-risk` label automatically.
+
+## Errors
+
+| Error | When | Resolution |
+|-------|------|------------|
+| Enlist failure | No opposing-provider backend available | PR marked as assessed; review proceeds without risk gate |
+| Dispatch failure | Context gathering (diff, files, issue) fails | Risk agent killed; PR marked as assessed |
+| Stall timeout | Risk agent doesn't respond within stall timeout | PR marked as assessed with `TIMEOUT` level |
+
+## Scope Boundary
+
+**In scope**: Risk agent dispatch, verdict parsing, label routing, stale assessment cleanup, objective resolution, issue linkage parsing.
+
+**Out of scope**: Reviewer dispatch (review package), label application (comment tool in mcp package), HITL escalation mechanics (review package).

package/packages/risk/src/index.js

@@ -0,0 +1,428 @@
+import { join, dirname } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { Workflow } from 'loreli/workflow';
+import { logger } from 'loreli/log';
+import { mark, has, parse } from 'loreli/marker';
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const log = logger('risk');
+
+/**
+ * Pattern matching GitHub's issue-closing keywords in PR bodies.
+ * Captures the issue number from "Closes #N", "Fixes #N", etc.
+ *
+ * @type {RegExp}
+ */
+const CLOSES_RE = /(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/i;
+
+/**
+ * Risk assessment workflow for Loreli's orchestration pipeline.
+ *
+ * Manages risk agents — dispatching them for new PRs, reading their
+ * verdicts, applying risk labels, and routing CRITICAL PRs to HITL.
+ * Runs before ReviewWorkflow in the reactor chain so labels are
+ * applied before scan() checks for them.
+ *
+ * The contract with ReviewWorkflow is GitHub labels:
+ * - `loreli:low-risk` — PR cleared for normal review
+ * - `loreli:medium-risk` — PR cleared with risk context warning
+ * - `loreli:critical-risk` — PR escalated to HITL, no reviewer dispatched
+ *
+ * @extends Workflow
+ */
+export class RiskWorkflow extends Workflow {
+  /** @type {string} Agent role this workflow manages. */
+  static role = 'risk';
+
+  /** @type {string} Mustache template path for risk prompts. */
+  static template = join(__dirname, '..', 'prompts', 'risk.md');
+
+  /**
+   * PRs awaiting risk assessment before reviewer dispatch.
+   * Keys are PR numbers, values are { riskAgent, dispatchedAt }.
+   *
+   * @type {Map<number, {riskAgent: string, dispatchedAt: number}>}
+   */
+  _assessing = new Map();
+
+  /**
+   * PRs that have completed risk assessment (verdict received or timed out).
+   * Prevents assess() from re-dispatching risk agents for already-handled PRs.
+   *
+   * @type {Set<number>}
+   */
+  _assessed = new Set();
+
+  /**
+   * Report risk demand: how many PRs need risk assessment vs how many
+   * risk agents are active. Uses hydrated state so no extra API calls.
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @returns {Promise<{workload: number, supply: number, deficit: number}>}
+   */
+  async demand(repo) {
+    const skip = this.orchestrator.cfg?.get?.('review.skipRiskAssessment') ?? false;
+    if (skip) return { workload: 0, supply: 0, deficit: 0 };
+
+    const prs = await this.hub.pulls(repo, { state: 'open' });
+
+    let workload = 0;
+    for (const pr of prs) {
+      if (this._assessing.has(pr.number)) continue;
+      if (this._assessed.has(pr.number)) continue;
+
+      const hasRiskLabel = pr.labels?.some(function isRisk(l) {
+        const name = l.name ?? l;
+        return name.endsWith('-risk') && name.startsWith('loreli:');
+      });
+      if (hasRiskLabel) continue;
+
+      workload++;
+    }
+
+    const supply = this._assessing.size;
+
+    return { workload, supply, deficit: Math.max(0, workload - supply) };
+  }
+
+  /**
+   * Register reactor handlers for the orchestrator's tick loop.
+   * Hydrate → Assess → Verdict → Reap runs sequentially on each tick,
+   * before ReviewWorkflow's scan() so labels are applied first.
+   *
+   * @returns {Record<string, function>} Handler map.
+   */
+  reactor() {
+    const self = this;
+    return {
+      async 'risk-hydrate'(repo) { await self.hydrate(repo); },
+      async assess(repo) { await self.assess(repo); },
+      async verdict(repo) { await self.verdict(repo); },
+      async 'risk-reap'(repo) { await self.reap(repo); }
+    };
+  }
+
+  /**
+   * Rehydrate in-memory risk state from GitHub artifacts.
+   *
+   * Called at the start of each reactor tick so that PRs assessed or
+   * being assessed by other participants are visible to verdict() and
+   * reap(). Without this, a process that restarts (or a second
+   * participant) would miss PRs already in the assessment pipeline.
+   *
+   * Derives state from two sources:
+   * - `loreli:*-risk` labels → marks PR as assessed
+   * - `risk-claim` marker comments → marks PR as mid-assessment
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @returns {Promise<void>}
+   */
+  async hydrate(repo) {
+    if (!this.hub) return;
+
+    const prs = await this.hub.pulls(repo, { state: 'open' });
+
+    for (const pr of prs) {
+      if (this._assessing.has(pr.number) || this._assessed.has(pr.number)) continue;
+
+      const hasRiskLabel = pr.labels?.some(function isRisk(l) {
+        const name = l.name ?? l;
+        return name.endsWith('-risk') && name.startsWith('loreli:');
+      });
+      if (hasRiskLabel) {
+        this._assessed.add(pr.number);
+        continue;
+      }
+
+      const comments = await this.hub.comments(repo, pr.number);
+      const riskClaim = comments.find(function isClaim(c) { return has(c.body, 'risk-claim'); });
+      if (!riskClaim) continue;
+
+      // Verdict already posted → assessed
+      const hasVerdict = comments.some(function isVerdict(c) { return has(c.body, 'risk'); });
+      if (hasVerdict) {
+        this._assessed.add(pr.number);
+      } else {
+        const agentName = parse(riskClaim.body, 'risk-claim')?.agent;
+        this._assessing.set(pr.number, {
+          riskAgent: agentName ?? 'unknown',
+          dispatchedAt: new Date(riskClaim.created ?? riskClaim.created_at).getTime()
+        });
+      }
+    }
+  }
+
+  // ── Helpers ───────────────────────────────────────────
+
+  /**
+   * Extract the first "Closes #N" or "Fixes #N" issue number from a body.
+   *
+   * @param {string} body - PR or issue body.
+   * @returns {number|null} Issue number, or null if not found.
+   */
+  closesIssue(body) {
+    if (!body) return null;
+    const m = body.match(CLOSES_RE);
+    return m ? parseInt(m[1], 10) : null;
+  }
+
+  /**
+   * Resolve the planning objective from the parent issue.
+   * Finds parent via loreli:parent label and sub-issue linkage.
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @param {number} childNumber - Child issue number (from Closes #N).
+   * @returns {Promise<string>} Objective string, or fallback.
+   */
+  async objective(repo, childNumber) {
+    const parents = await this.hub.issues(repo, { state: 'open', labels: ['loreli:parent'] });
+    const confirmed = parents.filter(function withMarker(i) { return has(i.body, 'parent'); });
+
+    for (const parent of confirmed) {
+      let subs;
+      try {
+        subs = await this.hub.subs(repo, parent.number);
+      } catch {
+        continue;
+      }
+      const isChild = subs.some(function match(s) { return s.number === childNumber; });
+      if (isChild) {
+        const data = parse(parent.body, 'parent');
+        return data?.objective ?? 'Planning objective';
+      }
+    }
+    return 'Planning objective';
+  }
+
+  // ── Assess ───────────────────────────────────────────
+
+  /**
+   * Scan for new PRs from action agents and dispatch risk agents.
+   * Only targets PRs that haven't been assessed yet and don't already
+   * have a risk label. Gathers diff, file stats, issue body, and
+   * planning objective as context for the risk agent.
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @returns {Promise<Array<{pr: number, agent: string}>>}
+   */
+  async assess(repo) {
+    const skip = this.orchestrator.cfg?.get?.('review.skipRiskAssessment') ?? false;
+    if (skip) return [];
+
+    const dispatched = [];
+    const prs = await this.hub.pulls(repo, { state: 'open' });
+    const actions = [...this.orchestrator.agents.values()]
+      .filter(function isAction(a) { return a.role === 'action'; });
+
+    for (const pr of prs) {
+      if (this._assessing.has(pr.number)) continue;
+      if (this._assessed.has(pr.number)) continue;
+
+      // Skip PRs that already have a risk label (from a previous assessment)
+      const hasRiskLabel = pr.labels?.some(function isRisk(l) {
+        const name = l.name ?? l;
+        return name.endsWith('-risk') && name.startsWith('loreli:');
+      });
+      if (hasRiskLabel) continue;
+
+      const agent = actions.find(function owns(a) {
+        return pr.head.startsWith(a.identity.name + '/');
+      });
+      if (!agent) continue;
+
+      // Verify a risk agent is available BEFORE claiming. Claiming
+      // without an available agent creates a deadlock: hydrate() sees the
+      // claim on the next tick, adds the PR to _assessing, and assess()
+      // skips it forever — but no agent was dispatched.
+      const riskAgents = this.agents().filter(function ready(a) {
+        return a.state === 'spawned' || a.state === 'working';
+      });
+      const assessing = this._assessing;
+      const riskAgent = riskAgents.find(function free(a) {
+        return ![...assessing.values()].some(function busy(t) { return t.riskAgent === a.identity.name; });
+      });
+      if (!riskAgent) {
+        log.debug(`assess: no risk agent available for PR #${pr.number} — waiting for scale()`);
+        continue;
+      }
+
+      // Check for existing risk-claim marker — another participant
+      // may have already dispatched a risk agent for this PR.
+      const prComments = await this.hub.comments(repo, pr.number);
+      if (prComments.some(function isClaimed(c) { return has(c.body, 'risk-claim'); })) continue;
+
+      // Optimistic claim: post a risk-claim marker as the risk agent, then verify first.
+      const visible = riskAgent.identity.claim?.() ?? `Claimed by **${riskAgent.identity.name}**`;
+      const won = await this.claimFirst(repo, pr.number, 'risk-claim', riskAgent.identity, riskAgent.role, visible);
+      if (!won) {
+        log.info(`assess: lost risk-claim race for PR #${pr.number} — skipping`);
+        continue;
+      }
+
+      try {
+        const childNum = this.closesIssue(pr.body);
+        let issueBody = pr.body;
+        if (childNum && typeof this.hub.issue === 'function') {
+          const linked = await this.hub.issue(repo, childNum);
+          issueBody = linked?.body ?? pr.body;
+        }
+        const obj = childNum && typeof this.hub.issues === 'function' && typeof this.hub.subs === 'function'
+          ? await this.objective(repo, childNum)
+          : 'Planning objective';
+
+        const files = await this.hub.files(repo, pr.number);
+        let diff = '';
+        if (typeof this.hub.diff === 'function') {
+          diff = await this.hub.diff(repo, pr.number);
+        }
+
+        await this.saveTask(riskAgent.identity.name, {
+          type: 'assess_risk',
+          pr: pr.number
+        });
+
+        const prompt = await this.render({
+          objective: obj,
+          issue: issueBody,
+          number: pr.number,
+          title: pr.title,
+          head: pr.head,
+          base: pr.base,
+          author: agent.identity.name,
+          authorProvider: agent.identity.provider,
+          files: files.map(function fmt(f) {
+            return {
+              filename: f.filename,
+              status: f.status,
+              additions: f.additions ?? 0,
+              deletions: f.deletions ?? 0
+            };
+          }),
+          diff
+        });
+
+        await riskAgent.send(prompt);
+        this.orchestrator.activity(riskAgent.identity.name);
+        this._assessing.set(pr.number, {
+          riskAgent: riskAgent.identity.name,
+          dispatchedAt: Date.now()
+        });
+        dispatched.push({ pr: pr.number, agent: riskAgent.identity.name });
+        log.info(`assess: dispatched ${riskAgent.identity.name} for PR #${pr.number}`);
+      } catch (err) {
+        log.warn(`assess: dispatch failed for PR #${pr.number}: ${err.message}`);
+        try { await this.orchestrator.kill(riskAgent.identity.name); } catch { /* best-effort */ }
+
+        // Apply fallback low-risk label so review's label gate passes
+        try {
+          const label = 'loreli:low-risk';
+          await this.hub.ensure(repo, [{ name: label, color: '0e8a16', description: 'Risk: LOW' }]);
+          await this.hub.label(repo, pr.number, [label]);
+          log.info(`assess: applied fallback ${label} to PR #${pr.number}`);
+        } catch { /* best-effort — review may still proceed on next scan */ }
+
+        this._assessed.add(pr.number);
+      }
+    }
+
+    return dispatched;
+  }
+
+  // ── Verdict ──────────────────────────────────────────
+
+  /**
+   * Check for risk verdicts on PRs being assessed. When a verdict
+   * comment is found, kills the risk agent and marks the PR as assessed.
+   * CRITICAL verdicts escalate to HITL via ReviewWorkflow.
+   *
+   * Labels are applied by the comment tool when the risk agent posts
+   * its verdict with `risk: true`. This handler just reads the result
+   * and routes accordingly.
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @returns {Promise<Array<{pr: number, level: string}>>}
+   */
+  async verdict(repo) {
+    if (!this._assessing.size) return [];
+
+    const results = [];
+    const stallTimeout = this.orchestrator.stallTimeout;
+    const now = Date.now();
+
+    for (const [prNum, tracking] of this._assessing) {
+      const elapsed = now - tracking.dispatchedAt;
+
+      // Stall timeout — risk agent didn't respond. Apply a fallback
+      // low-risk label so ReviewWorkflow.scan() can proceed — without
+      // a label the PR stays gated indefinitely.
+      if (elapsed > stallTimeout) {
+        log.warn(`verdict: risk assessment timed out for PR #${prNum} — applying low-risk fallback`);
+        this._assessing.delete(prNum);
+        this._assessed.add(prNum);
+        try { await this.orchestrator.kill(tracking.riskAgent); } catch { /* best-effort */ }
+
+        try {
+          const label = 'loreli:low-risk';
+          await this.hub.ensure(repo, [{ name: label, color: '0e8a16', description: 'Risk: LOW' }]);
+          await this.hub.label(repo, prNum, [label]);
+          log.info(`verdict: applied fallback ${label} to PR #${prNum}`);
+        } catch (err) {
+          log.warn(`verdict: failed to apply fallback label to PR #${prNum}: ${err.message}`);
+        }
+
+        results.push({ pr: prNum, level: 'TIMEOUT' });
+        continue;
+      }
+
+      const comments = await this.hub.comments(repo, prNum);
+      const riskComment = comments.find(function hasRisk(c) { return has(c.body, 'risk'); });
+      if (!riskComment) continue;
+
+      const data = parse(riskComment.body, 'risk');
+      const level = data?.level?.toUpperCase?.() ?? 'LOW';
+
+      try { await this.orchestrator.kill(tracking.riskAgent); } catch { /* best-effort */ }
+      this._assessing.delete(prNum);
+      this._assessed.add(prNum);
+
+      log.info(`verdict: PR #${prNum} assessed as ${level}`);
+      results.push({ pr: prNum, level });
+    }
+
+    return results;
+  }
+
+  // ── Reap ─────────────────────────────────────────────
+
+  /**
+   * Clean up stale risk assessments. Risk agents that die without
+   * posting a verdict leave PRs stuck in _assessing forever. This
+   * handler checks for dead agents and marks their PRs as assessed
+   * so ReviewWorkflow can proceed.
+   *
+   * @param {string} repo - Repository in "owner/name" format.
+   * @returns {Promise<void>}
+   */
+  async reap(repo) {
+    for (const [prNum, tracking] of this._assessing) {
+      const agent = this.orchestrator.agents.get(tracking.riskAgent);
+      if (!agent) {
+        log.info(`reap: risk agent ${tracking.riskAgent} gone — clearing PR #${prNum}`);
+        this._assessing.delete(prNum);
+        this._assessed.add(prNum);
+
+        // Apply fallback low-risk label so ReviewWorkflow's label gate
+        // can proceed. Without this, the PR enters a dead zone: assessed
+        // in memory but unlabeled on GitHub, permanently skipped by scan().
+        try {
+          const label = 'loreli:low-risk';
+          await this.hub.ensure(repo, [{ name: label, color: '0e8a16', description: 'Risk: LOW' }]);
+          await this.hub.label(repo, prNum, [label]);
+          log.info(`reap: applied fallback ${label} to PR #${prNum}`);
+        } catch (err) {
+          log.warn(`reap: failed to apply fallback label to PR #${prNum}: ${err.message}`);
+        }
+      }
+    }
+  }
+}