ai-foreman 1.7.17 → 1.7.19

package/README.md

@@ -15,9 +15,9 @@ Loop Claude Code or Codex through tickets.
 - **Rich ticket system** — tickets carry acceptance criteria, required tests, dependencies, priority, size, and risk level. The agent reads the full context before starting each step.
 - **Future work tracking** — when the builder discovers out-of-scope work during a run, `ai-foreman tickets discover` captures it without derailing the current ticket. Discovered items live in a separate inbox until you promote them.
 - **QA that actually gates** — QA runs after every completed ticket and checks code quality, test passing, and regression protection. The ticket only closes after QA passes. QA turns do not count against `--steps`.
-- **Visible long-running work** — semantic activity rows identify provider, phase, tools, retries, and quiet periods; cursor-capable terminals update changing counters in place, while record-oriented terminals coalesce them.
+- **Visible long-running work** — semantic activity rows identify provider, phase, tools, retries, and quiet periods; safe record output coalesces spinner- and time-only changes.
 
-Provider-managed retries are reported immediately as durable lines. A quiet-provider warning appears after 60 seconds without a signal, but Foreman does not add retries, abort the provider, or change provider retry limits. Non-TTY output receives a heartbeat every 30 seconds while work remains pending. TTY rendering defaults to automatic detection: Codex, CI, and `TERM=dumb` use newline records and suppress spinner- or number-only repeats; other terminals redraw those changes in place and retain a row when the textual status changes. Use `RAFI_ACTIVITY_RENDER_MODE=auto|cursor|records` to override detection.
+Provider-managed retries are reported immediately as durable lines. A quiet-provider warning appears after 60 seconds without a signal, but Foreman does not add retries, abort the provider, or change provider retry limits. Non-TTY output emits the initial and meaningful status changes without ANSI redraws. Automatic TTY output also uses newline records and suppresses spinner-, elapsed-time-, and number-only repeats, making it safe for transcript hosts. Set `RAFI_ACTIVITY_RENDER_MODE=cursor` to explicitly enable animated in-place redraws in a known real terminal; `records` explicitly forces safe records, while `auto` selects the safe default.
 
 ## Install
 
@@ -77,13 +77,14 @@ What each part does:
 
 - `./my-project`: the repo Foreman will work on.
 - `tickets init`: creates the local `.tickets/` tracker.
-- `tickets populate`: asks the ticket-maker role to convert project planning material.
+- `tickets populate`: generates tickets from an approved structured Rafi plan, with optional supporting context.
 - Converted output uses Foreman's schema.
 - `start`: drives Codex through the next tickets one step at a time.
 
-`tickets populate` can convert:
+In `approved_plan` mode, `tickets populate` requires a validated `<docs.root>/rafi-plan.json`. Supporting documents and URLs add detail but cannot replace its slices. `external_import` is a separate mode for importing configured Linear or Jira tickets.
+
+Supporting context can include:
 
-- a managed Rafi plan at `<docs.root>/rafi-plan.md`
 - plans
 - TODOs
 - roadmap docs
@@ -161,6 +162,16 @@ Saved `tickets.build` defaults in `rafi-config.yaml` can enable branch-per-ticke
 
 With the saved `current` strategy, Foreman works in the active branch while the user owns Git. It may edit, test, run QA, and update tracker/recovery state, but all branch/worktree, commit, push, merge, rebase, and review lifecycle commands are fenced. An unexpected active ref or worktree change pauses the run. Explicit isolated flags are reported as run overrides.
 
+Build stall recovery uses current context occupancy, not lifetime token usage. The `compact` session strategy checks occupancy at ordinary boundaries and compacts only when the threshold requires it; `fresh` still creates an intentional new session. Compaction normally has a 120-second deadline, extending to at most 180 seconds only after correlated progress. Late completion is attributed to the original session; unresolved attempts remain quarantined across restart. Unknown completion pauses recovery rather than replaying implementation in a replacement session.
+
+Unchanged, source-bound plan and ticket approvals are reused. Material ticket, plan, delivery, or workflow changes require a new decision, including with `--yes`. Legacy approval evidence that cannot prove unchanged scope requires review. `--steps` counts completed tickets or steps; it is neither a wall-clock limit nor a model-turn budget.
+
+Questions are persisted before prompting. Redirected or detached runs return a recoverable needs-input outcome (exit 2). Use `rafi build:attach . --run <id>` to see decisions, `rafi build:decide . --run <id> --decision <id> --choice <id>` to answer, and `rafi build:resume . --run <id>` to continue. Custom answers additionally use `--choice custom --answer "text"`. Independent eligible tickets continue around a ticket blocker; dependent and unsafe shared-delivery work stays deferred. Answered ticket questions are reconsidered at safe boundaries.
+
+CLI starts run under a durable parent supervisor by default. `--detach` writes output under `.rafi/logs/supervisor-<run>.log`; `--no-supervisor` retains direct execution. A preparation-only worker crash may restart within frozen budgets after proving the worker and registered readiness children are stopped. A worker crash with unresolved provider work pauses for reconciliation: it does not silently repeat a possibly completed action. `build:stop` signals only a verified supervisor process incarnation.
+
+Runtime deadlines can be overridden in `autonomy.runtime_deadlines` using positive integer milliseconds: `preparation_ms` (120000), `rpc_ms` (60000), `compaction_ms` (120000), `shutdown_ms` (10000), and `turn_ms` (3600000). They are frozen with the run policy. Transport-idle and hard active-turn deadlines have distinct meanings; neither grants permission to repeat uncertain side effects. Attached-process elapsed time includes waiting and is not provider compute time. Activity warns after five minutes without a completed tool or agent turn; intentional human waits are excluded, and the warning alone does not cancel work. Final paused output lists the durable question and exact run-scoped recovery commands.
+
 The current terminal status includes the role/provider, activity, truthful context state, successful compaction count, and handoff generation. At the Builder threshold, Foreman settles the in-flight action, verifies native compaction plus a fresh provider usage sample, and resumes the frozen action. The configured maximum defaults to ten successful compactions per location-scoped provider session; the next crossing uses a validated fresh handoff instead of exceeding it. Disposable QA snapshots never reuse a prior QA conversation in a new `/tmp/rafi-qa-*` worktree; they transfer cumulative state through an accepted handoff.
 
 Failed QA reviews use the versioned `RAFI_QA_FAILURE_REPORT_START` / `RAFI_QA_FAILURE_REPORT_END` JSON contract. Foreman validates the report before Builder remediation, namespaces finding IDs by run, ticket, and review, and durably indexes the exact report, remediation response, and bounded fix summary by digest. A malformed failing report receives two correction-only turns in its original session, then a validated fresh successor performs one complete source-bound review followed by up to two correction-only turns. Any later operator-requested fresh successor performs one complete source-bound review followed by five usable correction-only turns.
@@ -214,11 +225,11 @@ gh repo view <host>/<owner>/<repo>
 | `-a, --agent <agent>` | `claude` / `codex` | single `rafi-config.yaml` target, otherwise `claude` | Builder agent. Explicit `--agent` always wins. |
 | `-m, --model <model>` | `gpt-5.5` / any supported agent model | agent default | Overrides the builder model. |
 | `--effort <level>` | Claude Code: `low` / `medium` / `high` / `xhigh`<br>Codex: `low` / `medium` / `high` / `xhigh` | agent default | Reasoning level. |
-| `--sources <paths...>` | `docs/tickets.md docs/plans/**` | Configured Rafi `<docs.root>/rafi-plan.md`, then ticket docs root, then `docs/rafi-plan.md`, otherwise scans | Optional files, folders, or globs for the agent to check first. Any reasonable planning format is OK. |
+| `--sources <paths...>` | `docs/tickets.md docs/plans/**` | saved context, subject to `source_handling` | Supplemental files, folders, URLs, or globs. Passing an approved `rafi-plan.json` remains supported; ordinary documents cannot replace the approved plan. |
 | `--fast` | flag | off | Lower latency. |
 | `-y, --yes` | flag | off | Skips confirmation before builder edits ticket files. |
 
-Use `ai-foreman tickets setup:init` or `setup:update` to append local, Linear, or Jira Cloud sources to the project-wide registry in `rafi-config.yaml`. Linear imports use `LINEAR_API_KEY`; Jira Cloud imports use `JIRA_EMAIL` and `JIRA_API_TOKEN`. Credentials are never persisted, and mirrored tickets retain source provenance.
+Use `ai-foreman tickets setup:init` or `setup:update` to manage supporting local/URL context and external Linear/Jira connections in the project-wide registry in `rafi-config.yaml`. Linear imports use `LINEAR_API_KEY`; Jira Cloud imports use `JIRA_EMAIL` and `JIRA_API_TOKEN`. Credentials are never persisted, and mirrored tickets retain source provenance.
 
 Use `ai-foreman tickets review` to accept, defer, or dismiss pending split/combine/duplicate recommendations rendered in the progress doc.
 

package/dist/activity.d.ts

@@ -11,7 +11,8 @@ export interface ActivityReporterOptions {
     tickMs?: number;
     heartbeatMs?: number;
     quietWarningMs?: number;
-    /** Override automatic TTY capability detection for tests or embedding hosts. */
+    usefulProgressWarningMs?: number;
+    /** Override the safe automatic TTY rendering mode for tests or embedding hosts. */
     ttyMode?: ActivityTtyMode;
 }
 export type ActivityTtyMode = "auto" | "cursor" | "records";
@@ -24,6 +25,10 @@ export declare class ActivityReporter {
     private readonly tickMs;
     private readonly heartbeatMs;
     private readonly quietWarningMs;
+    private readonly usefulProgressWarningMs;
+    private lastUsefulProgressAt;
+    private usefulProgressWarningPrinted;
+    private pauseStartedAt;
     private readonly ttyMode;
     private readonly commandStartedAt;
     private readonly phases;
@@ -48,6 +53,8 @@ export declare class ActivityReporter {
         immediate?: boolean;
     }): void;
     pulse(detail?: string): void;
+    /** A completed tool or turn is distinct from transport/status activity. */
+    completedWork(): void;
     note(message: string): void;
     writePersistent(text: string): void;
     /** Keep role/provider/context state attached to the current activity status. */
@@ -71,5 +78,8 @@ export declare function pauseActivityForInput<T>(operation: () => Promise<T>, _m
 export declare function reportBuilderEvent(event: BuilderEvent): void;
 /** Async event queue that also feeds the active CLI reporter without consuming the stream. */
 export declare class BuilderEventQueue extends AsyncQueue<BuilderEvent> {
+    private readonly observers;
+    /** Synchronous fan-out; never consumes or competes with the owning iterator. */
+    observe(listener: (event: BuilderEvent) => void): () => void;
     push(event: BuilderEvent): void;
 }

package/dist/activity.js

@@ -10,6 +10,10 @@ export class ActivityReporter {
     tickMs;
     heartbeatMs;
     quietWarningMs;
+    usefulProgressWarningMs;
+    lastUsefulProgressAt = 0;
+    usefulProgressWarningPrinted = false;
+    pauseStartedAt = 0;
     ttyMode;
     commandStartedAt;
     phases = new Map();
@@ -37,6 +41,7 @@ export class ActivityReporter {
         this.tickMs = options.tickMs ?? 1_000;
         this.heartbeatMs = options.heartbeatMs ?? 30_000;
         this.quietWarningMs = options.quietWarningMs ?? 60_000;
+        this.usefulProgressWarningMs = options.usefulProgressWarningMs ?? 300_000;
         this.ttyMode = resolveTtyMode(options.ttyMode);
         this.commandStartedAt = this.now();
     }
@@ -45,6 +50,8 @@ export class ActivityReporter {
         this.phases.set(phase.id, phase);
         if (this.phases.size === 1) {
             this.lastSignalAt = phase.startedAt;
+            this.lastUsefulProgressAt = phase.startedAt;
+            this.usefulProgressWarningPrinted = false;
             this.lastHeartbeatAt = phase.startedAt;
             this.quietWarningPrinted = false;
             this.startTimer();
@@ -80,6 +87,11 @@ export class ActivityReporter {
         this.quietWarningPrinted = false;
         this.render(this.output.isTTY);
     }
+    /** A completed tool or turn is distinct from transport/status activity. */
+    completedWork() {
+        this.lastUsefulProgressAt = this.now();
+        this.usefulProgressWarningPrinted = false;
+    }
     note(message) {
         this.clearLine();
         this.output.write(`${clean(message, 500)}\n`);
@@ -109,6 +121,8 @@ export class ActivityReporter {
         this.render(true);
     }
     pause() {
+        if (this.paused === 0)
+            this.pauseStartedAt = this.now();
         this.paused++;
         this.clearLine();
         let resumed = false;
@@ -117,6 +131,11 @@ export class ActivityReporter {
                 return;
             resumed = true;
             this.paused = Math.max(0, this.paused - 1);
+            if (this.paused === 0) {
+                const duration = this.now() - this.pauseStartedAt;
+                this.lastUsefulProgressAt += duration;
+                this.lastSignalAt += duration;
+            }
             this.render(true);
         };
     }
@@ -161,9 +180,17 @@ export class ActivityReporter {
         if (!force && elapsed < this.displayDelayMs)
             return;
         const quietFor = now - this.lastSignalAt;
+        if (phase && now - this.lastUsefulProgressAt >= this.usefulProgressWarningMs && !this.usefulProgressWarningPrinted) {
+            this.usefulProgressWarningPrinted = true;
+            this.note(`rafi: no completed tool or agent turn observed for ${formatDuration(now - this.lastUsefulProgressAt)}; current phase: ${phase.label}. The operation remains subject to its configured deadline.`);
+            return;
+        }
         if (phase && quietFor >= this.quietWarningMs && !this.quietWarningPrinted) {
             this.quietWarningPrinted = true;
-            this.note(`rafi: ${this.provider ?? "provider"} has been quiet for ${formatDuration(quietFor)}; RAFI is still responsive and will keep waiting`);
+            const status = phase.label === "answer sent; waiting for Claude"
+                ? `Claude has not resumed its stream yet after your answer was sent (${formatDuration(quietFor)})`
+                : `${this.provider ?? "provider"} has been quiet for ${formatDuration(quietFor)}`;
+            this.note(`rafi: ${status}; RAFI is still responsive and will keep waiting`);
             return;
         }
         const provider = [this.provider, this.model].filter(Boolean).join("/");
@@ -188,9 +215,16 @@ export class ActivityReporter {
             this.lastSemanticKey = semanticKey;
             return;
         }
+        const semanticKey = semanticStatusKey(body);
+        // Append-only outputs cannot replace an earlier line.  Emit the first
+        // state and meaningful state changes, but do not turn a changing counter
+        // or elapsed time into an endless log stream.
+        if (semanticKey === this.lastSemanticKey)
+            return;
         if (force || now - this.lastHeartbeatAt >= this.heartbeatMs) {
             this.lastHeartbeatAt = now;
             this.output.write(`[${new Date(now).toISOString()}] rafi working: ${body} (${formatDuration(now - this.commandStartedAt)})\n`);
+            this.lastSemanticKey = semanticKey;
         }
     }
     clearLine() {
@@ -246,22 +280,49 @@ export function reportBuilderEvent(event) {
         reporter.note(`rafi: ${event.provider} ${clean(event.reason, 300)}; retrying${attempt}${delay}`);
         reporter.update(`retrying ${event.provider}`, event.reason, { provider: event.provider });
     }
-    else if (event.kind === "tool")
-        reporter.update(`running ${event.name}`, briefInput(event.input));
+    else if (event.kind === "tool") {
+        switch (event.lifecycle) {
+            case "progress":
+                reporter.pulse();
+                break;
+            case "completed":
+                reporter.completedWork();
+                if (isFailedToolCompletion(event))
+                    reporter.update("tool failed", event.name);
+                else
+                    reporter.pulse();
+                break;
+            case "started":
+            case undefined:
+                // Older event producers omitted lifecycle for a tool start.
+                reporter.update(`running ${event.name}`, briefInput(event.input));
+                break;
+        }
+    }
     else if (event.kind === "text")
         reporter.update("processing agent response");
     else if (event.kind === "session-transition")
         reporter.update(`session ${event.transition}`, event.detail);
     else if (event.kind === "context-usage")
         reporter.pulse(event.percentage === undefined ? undefined : `context ${event.percentage.toFixed(0)}%`);
-    else if (event.kind === "turn-complete")
+    else if (event.kind === "turn-complete") {
+        reporter.completedWork();
         reporter.update(event.result.isError ? "agent turn failed" : "agent turn complete");
+    }
     else if (event.kind === "error")
         reporter.update("provider error", event.message);
 }
 /** Async event queue that also feeds the active CLI reporter without consuming the stream. */
 export class BuilderEventQueue extends AsyncQueue {
+    observers = new Set();
+    /** Synchronous fan-out; never consumes or competes with the owning iterator. */
+    observe(listener) {
+        this.observers.add(listener);
+        return () => { this.observers.delete(listener); };
+    }
     push(event) {
+        for (const listener of this.observers)
+            listener(event);
         reportBuilderEvent(event);
         super.push(event);
     }
@@ -273,30 +334,37 @@ function briefInput(input) {
     const detail = value.command ?? value.file_path ?? value.path ?? value.pattern ?? value.query;
     return detail === undefined ? undefined : clean(String(detail), 100);
 }
+function isFailedToolCompletion(event) {
+    const status = event.status?.toLowerCase();
+    return status === "failed"
+        || status === "error"
+        || (typeof event.exitCode === "number" && Number.isFinite(event.exitCode) && event.exitCode !== 0);
+}
 function clean(value, maximum) {
     return value.replace(/[\r\n\t]+/g, " ").replace(/\s+/g, " ").trim().slice(0, maximum);
 }
 function semanticStatusKey(body) {
-    return body.replace(/\p{Number}+(?:[.,]\p{Number}+)*/gu, "#");
+    return body
+        // Counters, percentages, ticket suffixes, and model version numbers.
+        .replace(/\p{Number}+(?:[.,]\p{Number}+)*/gu, "#")
+        // Commit IDs, request IDs, and generated hexadecimal identifiers.
+        .replace(/\b[\da-f]{7,}\b/giu, "#")
+        .replace(/\s+/g, " ")
+        .trim();
 }
 function resolveTtyMode(requested) {
-    const configured = requested ?? configuredTtyMode(process.env.RAFI_ACTIVITY_RENDER_MODE);
+    const configured = requested === "cursor" || requested === "records"
+        ? requested
+        : configuredTtyMode(process.env.RAFI_ACTIVITY_RENDER_MODE);
     if (configured === "cursor" || configured === "records")
         return configured;
-    return truthyEnvironmentValue(process.env.CODEX_CI)
-        || truthyEnvironmentValue(process.env.CI)
-        || process.env.TERM?.toLowerCase() === "dumb"
-        ? "records"
-        : "cursor";
+    // A TTY-like output can still be an append-only transcript.  Cursor redraws
+    // are therefore an explicit opt-in rather than an unsafe capability guess.
+    return "records";
 }
 function configuredTtyMode(value) {
     return value === "cursor" || value === "records" || value === "auto" ? value : "auto";
 }
-function truthyEnvironmentValue(value) {
-    if (value === undefined)
-        return false;
-    return !["", "0", "false"].includes(value.trim().toLowerCase());
-}
 function formatDuration(milliseconds) {
     const seconds = Math.max(0, Math.floor(milliseconds / 1_000));
     if (seconds < 60)

package/dist/adapters/claude.d.ts

@@ -58,6 +58,8 @@ export declare class ClaudeAdapter implements BuilderAdapter {
     private structuredError?;
     private apiErrorStatus?;
     private compactResult?;
+    private manualCompactionDeadline?;
+    private compactionPromise?;
     private autoCompactionPrepared;
     private preparedAutoCompactThreshold?;
     private preparedAutoCompactionPolicy?;
@@ -70,6 +72,8 @@ export declare class ClaudeAdapter implements BuilderAdapter {
     private activeProviderTurnSpanId?;
     private readonly toolSpans;
     private pending?;
+    private questionTrace?;
+    private responseOnlyTurn;
     private terminalResult?;
     private streamEnded;
     private closed;
@@ -78,7 +82,11 @@ export declare class ClaudeAdapter implements BuilderAdapter {
     /** Background loop: consume the SDK message stream until it ends. */
     private pump;
     private handle;
-    sendTurn(text: string): Promise<TurnResult>;
+    sendTurn(text: string, policy?: {
+        handback?: boolean;
+        responseOnly?: boolean;
+        logicalActionId?: string;
+    }): Promise<TurnResult>;
     private sendTurnInternal;
     private captureStderr;
     private observationContext;
@@ -91,16 +99,23 @@ export declare class ClaudeAdapter implements BuilderAdapter {
     compact(): Promise<CompactResult>;
     private compactInternal;
     prepareAutoCompaction(thresholdPercent?: number | undefined): Promise<NativeAutoCompactionPolicy | void>;
+    private prepareAutoCompactionInternal;
     autoCompactionPolicy(): NativeAutoCompactionPolicy | undefined;
     drainNativeCompactions(): NativeCompaction[];
     restoreNativeCompactions(compactions: NativeCompaction[]): void;
     contextUsage(): Promise<ContextUsage | undefined>;
     sessionUsage(): Promise<ProviderSessionUsage | undefined>;
     switchSettings(settings: ProviderSettingSwitch): Promise<CompactResult>;
+    observeEvents(listener: (event: BuilderEvent) => void): () => void;
     events(): AsyncIterable<BuilderEvent>;
     close(): Promise<void>;
     private failSessionIdentity;
     private observeSession;
     private settlePending;
+    private touchPendingTurn;
+    private beginProviderQuestionWait;
+    private endProviderQuestionWait;
+    private armPendingTurn;
+    private providerIdleTimeoutMs;
     private streamFailureResult;
 }