@gr8ful/spf 0.19.0 → 0.19.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +8 -0
  2. package/assets/skill/references/config.md +1 -0
  3. package/dist/cli/commands/doctor.js +25 -3
  4. package/dist/cli/commands/estimate.d.ts +22 -6
  5. package/dist/cli/commands/estimate.js +32 -10
  6. package/dist/cli/commands/loop.d.ts +20 -0
  7. package/dist/cli/commands/loop.js +20 -1
  8. package/dist/cli/commands/ui.js +2 -1
  9. package/dist/cli/commands/watch.js +1 -1
  10. package/dist/cli/index.js +2 -2
  11. package/dist/cli/interview.js +13 -0
  12. package/dist/cli/ui/run_dashboard.js +13 -7
  13. package/dist/core/agent_cc.d.ts +19 -3
  14. package/dist/core/agent_cc.js +38 -18
  15. package/dist/core/agent_flue.js +51 -14
  16. package/dist/core/agent_opencode.d.ts +62 -25
  17. package/dist/core/agent_opencode.js +71 -30
  18. package/dist/core/agents.d.ts +51 -4
  19. package/dist/core/agents.js +79 -4
  20. package/dist/core/console.d.ts +24 -4
  21. package/dist/core/console.js +20 -7
  22. package/dist/core/data_types.d.ts +300 -19
  23. package/dist/core/data_types.js +134 -5
  24. package/dist/core/issues/jira_provider.d.ts +51 -1
  25. package/dist/core/issues/jira_provider.js +69 -1
  26. package/dist/core/issues/provider.d.ts +23 -0
  27. package/dist/core/loop.d.ts +39 -1
  28. package/dist/core/loop.js +33 -2
  29. package/dist/core/ollama_provider.d.ts +96 -13
  30. package/dist/core/ollama_provider.js +172 -26
  31. package/dist/core/otel.js +10 -1
  32. package/dist/core/otel_propagation.d.ts +168 -24
  33. package/dist/core/otel_propagation.js +219 -43
  34. package/dist/core/permissions.d.ts +16 -1
  35. package/dist/core/permissions.js +91 -3
  36. package/dist/core/providers.js +8 -3
  37. package/dist/core/refine.js +13 -1
  38. package/dist/core/runner.d.ts +33 -2
  39. package/dist/core/runner.js +40 -5
  40. package/dist/core/tiering.js +7 -3
  41. package/dist/core/tracer.d.ts +7 -1
  42. package/dist/core/tracer.js +15 -3
  43. package/dist/ui/server/db.d.ts +8 -1
  44. package/dist/ui/server/db.js +21 -4
  45. package/dist/ui/server/serve.d.ts +7 -0
  46. package/dist/ui/server/serve.js +10 -7
  47. package/dist/ui/shared/types.d.ts +16 -0
  48. package/package.json +1 -1
package/README.md CHANGED
@@ -240,6 +240,10 @@ ANTHROPIC_CUSTOM_HEADERS=cf-aig-authorization: Bearer <CF_AIG_TOKEN>
240
240
 
241
241
  Agent isolation in SPF is enforced **after the fact**, not upfront. This matters because the `tools:` capability list is not a sandbox — a `bash` tool can run `git checkout` to discard changes, and a `write` tool can reach any path, regardless of the allowlist you write. So permission is verified the way every other claim in this system is: the working tree is fingerprinted before an agent runs, then compared after. If the agent touched anything outside its `writes:` allowlist (or a `protected_files:` path it wasn't given access to), the phase fails and anything the agent *introduced* outside its allowlist is rolled back. What it cannot undo, it names: a file that was already dirty before the agent ran is left alone rather than discarded, and uncommitted work the agent reverted cannot be restored. And because the check is post-hoc and tree-scoped, nothing outside the working tree is in scope at all — a push, a network call, a write above `repo_root`. When running an agent on the claude_code backend, SPF spawns it with `--permission-mode bypassPermissions --dangerously-skip-permissions` because the backend has no need to enforce — SPF will enforce in code instead, within those bounds.
242
242
 
243
+ One narrow, explicit exception: `defaults.read_only_ignore` (default: the lockfiles of npm/pnpm/yarn/bun) names paths a **true read-only** agent (`writes: []`) may touch without failing the phase — a read-only agent that runs a dependency-manager command to inspect a package's real shape rewrites its lockfile as a side effect of a *read*, not an edit. The path is still always rolled back; the phase only *passes* when the path was clean before the phase started (so the rollback is a real restore) — a path already dirty when the phase began fails exactly like any other breach, whether the agent left it modified or reverted an operator's own uncommitted work. And the exemption is scoped to true read-only roles only: a write-restricted agent (`writes: [...]`) that changes a lockfile alongside something it *was* given (say `package.json`) left an inconsistent tree, not incidental bookkeeping, and fails like any other breach.
244
+
245
+ `protected_files` always takes precedence over `read_only_ignore`: a path matching `protected_files` is never exempt, even for a true read-only agent, even when it also happens to match a `read_only_ignore` pattern. `read_only_ignore` is a narrow "this churn is incidental" carve-out for dependency-manager bookkeeping — it is never a backdoor around a path an operator explicitly locked down with `protected_files`. That path is still rolled back like any ignored churn, but it fails the phase like any other breach.
246
+
243
247
  For unattended work, `spf watch`'s PR-merge is the only human accountability checkpoint: the daemon opens a PR for each automated run, and a human approves the merge. For any other workflow, this enforcement is the backbone of letting a bounded agent proposal survive code's inspection without a human having to read it in between — but it is bounded, not a sandbox, and the scope above is what a reviewer should actually rely on. (Attended `simple-sdlc` runs get a second checkpoint of their own — see below.)
244
248
 
245
249
  See `core/permissions.ts` for the implementation — the snapshot/compare logic, not the config terms.
@@ -821,6 +825,8 @@ Only the states you map are touched; an unmapped state keeps today's label-only
821
825
 
822
826
  `spf watch init` still doesn't create any labels here (Jira labels are freeform strings with no color/description registry to seed, unlike GitHub's) — it reports the labels this run will use. With `watch.refine.enabled`, it also validates `watch.jira.issue_types` against the real project's issue types, read-only, and exits non-zero on a mismatch — see "Refining specs" above. Independent of `refine.enabled`, if `watch.jira.status_map` has any entry configured, both `spf watch init` and `spf watch`'s own startup check validate it against the real project's statuses the same way, and exit non-zero on a mismatch.
823
827
 
828
+ When refining a spec into a tree, the published root(s) get linked back to the spec they came from via `watch.jira.link_type` (default `"Relates"`), a plain Jira issue link — never the hierarchical `parent` field, since the spec's own issue type frequently can't legally parent a root node's type. **`link_type` must name a symmetric link type**: the code fixes which side is which (the published root is always `inwardIssue`, the spec always `outwardIssue`) rather than exposing direction as a separate config knob, which is harmless for a symmetric type like "Relates" but would silently assert the opposite relationship for a directional one (e.g. "blocks"). If the issue-link API itself is unavailable (a renamed/removed link type, a restrictive permission scheme), it falls back to a plain comment naming the spec — and if even that fails, it's logged and swallowed rather than failing the whole publish over what is, at bottom, a cosmetic cross-reference.
829
+
824
830
  ### Bitbucket (`code_host: bitbucket`)
825
831
 
826
832
  ```bash
@@ -911,6 +917,8 @@ Full field reference: `spf install-skill`'s installed skill
911
917
 
912
918
  Every run produces a complete trace: all events, phases, agent calls, and tool invocations stream into SQLite as they happen. By default the trace is local-only and stays the source of truth — prompts, envelopes, tool arguments, and your source code never leave the machine. Token counts and costs ride alongside.
913
919
 
920
+ **Billable vs. total tokens.** Every token figure SPF reports is one of two numbers, and they diverge a lot on a long-running, prompt-caching session: the **display total** (`sessions.total_tokens`, the sessions panel's "tokens" line) counts every token re-sent on every turn, cache reads included — a 49-turn conversation over an 86k-token context bills millions of these even though only a fraction of that ever moved for the first time. The **billable** figure (`billable_tokens`) is `input + cache-write + output` only — material that actually entered the context for the first time or was newly generated — which is what `defaults.max_run_tokens` and `spf loop`'s `--max-tokens` are actually checked against, what `spf estimate`'s cutoff-phase projection walks, and what the live run dashboard compares against the ceiling (printing the display total alongside it, labeled "context incl. cache reads", for occupancy). A trace db from before this split existed just has `billable_tokens = 0` on every old row; every consumer that reads it back falls through to the display total for such a row rather than reporting a false zero.
921
+
914
922
  ```bash
915
923
  spf ui # browser-based visualizer over the trace
916
924
  spf events <adw_id> --follow # live event stream, tailable
@@ -82,6 +82,7 @@ agents:
82
82
  | `data_dir` | path | Runtime home, repo-relative. Default `.spf/data`. |
83
83
  | `max_run_cost` | number > 0 (USD) | Stops the NEXT agent call once a run has already spent this much — checked before each call, never after. A single call is never capped (a run can overshoot by one whole call), and a chain with only one agent dispatch (`scout`, `prompt`, `build`) can never trip it at all. Absent (default) = unbounded. Throws `BudgetExceeded`, which fails the phase closed. |
84
84
  | `max_run_tokens` | integer > 0 | Same semantics as `max_run_cost`, on `sessions.total_tokens` instead of cost. Absent (default) = unbounded. |
85
+ | `request_timeout_ms` | integer > 0 | `flue`-backend-only (maps onto `AgentStatics.durability.timeoutMs`; silently ignored for `claude_code`/`opencode`, which have no such knob). Bounds ONE agent dispatch, not the accumulated run. Absent (default) = Flue's own default applies unchanged (1 hour, 10 attempts) — a connection that dies silently mid-call hangs that long before anything notices. Not a precise deadline: Flue's own timeout check runs on a coarser periodic sweep in practice. Not back-filled onto agents — process-scoped, like `max_run_cost`/`max_run_tokens` above. |
85
86
 
86
87
  ### `spf fanout`
87
88
 
@@ -18,7 +18,7 @@ import * as agentOpencode from "../../core/agent_opencode.js";
18
18
  import { DEFAULT_NOTIFY_ENV_KEY } from "../../core/notify/notifier.js";
19
19
  import { endpointLabel, redact, resolveTracesUrl } from "../../core/otel.js";
20
20
  import { isKnownToolName as isKnownFlueToolName, resolveModel } from "../../core/agent_flue.js";
21
- import { ollamaBaseUrl } from "../../core/ollama_provider.js";
21
+ import { ollamaApiKey, ollamaBaseUrl } from "../../core/ollama_provider.js";
22
22
  import { cloudflareAiBaseUrl } from "../../core/cloudflare_provider.js";
23
23
  import { binaryOnPath, parseCli } from "../../core/utils.js";
24
24
  import { PROVIDER_ENV_KEYS } from "../../core/providers.js";
@@ -494,9 +494,31 @@ export async function doctorCommand(argv) {
494
494
  // latency either way — strictly cheaper for a pure reachability check,
495
495
  // and there's no live-server dependency in this choice: doctor's probe
496
496
  // itself tolerates either endpoint being down (see `probeGet`).
497
- const result = await withProbeStatus("OLLAMA_BASE_URL reachability", () => probeGet(`${ollamaBase}/models`));
497
+ //
498
+ // MINOR-F (MINOR 3: now via ollamaApiKey(), not a raw env read) — send a
499
+ // bearer on every probe, same as the Cloudflare Workers AI probe below
500
+ // AND same as a real dispatch: `ollamaApiKey()` (ollama_provider.ts) is
501
+ // the SAME function `registerOllamaModel`'s `auth.apiKey.resolve()`
502
+ // calls, so this probe and a real dispatch send byte-identical bearers
503
+ // for the same `OLLAMA_API_KEY` env state — including the dummy
504
+ // placeholder when it's unset, which a prior version of this probe
505
+ // omitted entirely (no `Authorization` header at all), a real
506
+ // divergence from what dispatch actually sends. A bare local Ollama
507
+ // server (the common case) checks nothing and answers identically
508
+ // either way; a gateway in front of it (e.g. Briefs' Envoy AI Gateway)
509
+ // 401/403s an unauthenticated, dummy, or wrong-key probe, which is
510
+ // exactly the misconfiguration doctor exists to surface, not silently
511
+ // mask as a generic "reachable: HTTP 401".
512
+ const ollamaApiKeyEnv = (process.env["OLLAMA_API_KEY"] ?? "").trim();
513
+ const ollamaHeaders = { authorization: `Bearer ${ollamaApiKey()}` };
514
+ const result = await withProbeStatus("OLLAMA_BASE_URL reachability", () => probeGet(`${ollamaBase}/models`, ollamaHeaders));
515
+ const isAuthFailure = result.ok && (result.status === 401 || result.status === 403);
498
516
  check(report, "OLLAMA_BASE_URL reachability", true, // informational/warning only — see the ANTHROPIC_BASE_URL check above for why
499
- result.ok ? `reachable: GET ${ollamaBase}/models -> HTTP ${result.status}` : `unreachable: GET ${ollamaBase}/models -> ${result.error}`, result.ok ? "info" : "warn");
517
+ !result.ok
518
+ ? `unreachable: GET ${ollamaBase}/models -> ${result.error}`
519
+ : isAuthFailure
520
+ ? `HTTP ${result.status} from GET ${ollamaBase}/models — a gateway in front of Ollama is rejecting this request; set OLLAMA_API_KEY to the client key it expects${ollamaApiKeyEnv ? " (one is set, but was rejected — check its value)" : " (none is currently set — the dummy placeholder bearer a keyless dispatch sends was rejected too)"}`
521
+ : `reachable: GET ${ollamaBase}/models -> HTTP ${result.status}`, !result.ok || isAuthFailure ? "warn" : "info");
500
522
  }
501
523
  // Cloudflare Workers AI — the same reachability check the Ollama block
502
524
  // above runs, for the same reason: a keyed provider whose endpoint
@@ -19,15 +19,19 @@ export declare function median(values: number[]): number;
19
19
  export declare function normalizePhaseName(name: string): string;
20
20
  export interface PhaseProjection {
21
21
  name: string;
22
+ /** Display total (cache reads included) — for the printed "phase p50 tokens" breakdown, i.e. context, never the ceiling comparison below. */
22
23
  p50: number;
23
24
  min: number;
24
25
  max: number;
26
+ /** BILLABLE p50 — what `findCutoffPhase` actually walks against `defaults.max_run_tokens` (a billable ceiling; see `UsageBreakdown.billable_tokens`'s doc comment in `core/data_types.ts`). Kept alongside, never instead of, `p50`: the two answer different questions and printing only one would silently pick a metric for the reader. */
27
+ billable_p50: number;
25
28
  }
26
29
  /**
27
- * Per run: sum every iteration of a normalized phase into one observation.
28
- * Across runs: median (p50) and min/max of those per-run sums. A phase
29
- * absent from a given run (a fix loop that never triggered) counts as a
30
- * `0` observation for that run — not a skip so its median reflects how
30
+ * Per run: sum every iteration of a normalized phase into one observation
31
+ * both the display total and the billable figure. Across runs: median (p50)
32
+ * and min/max of the total sums, plus the median of the billable sums. A
33
+ * phase absent from a given run (a fix loop that never triggered) counts as
34
+ * a `0` observation for that run — not a skip — so its median reflects how
31
35
  * often the phase actually ran, not just its size when it did.
32
36
  */
33
37
  export declare function aggregatePhases(sessions: ChainHistorySession[]): PhaseProjection[];
@@ -36,9 +40,21 @@ export interface RunTotalsProjection {
36
40
  min: number;
37
41
  max: number;
38
42
  }
39
- /** The HEADLINE: median of sampled runs' `sessions.total_tokens` — NOT the sum of the per-phase medians (those are different numbers; median of sums != sum of medians). */
43
+ /** The HEADLINE: median of sampled runs' `sessions.total_tokens` — NOT the sum of the per-phase medians (those are different numbers; median of sums != sum of medians). This is the CONTEXT figure (cache reads included); it is never compared against `max_run_tokens` — see `findCutoffPhase` for the billable-based ceiling check. */
40
44
  export declare function projectRunTotals(sessions: ChainHistorySession[]): RunTotalsProjection | null;
41
- /** Where a p50 run would be cut off by `max_run_tokens`, walking the phase breakdown cumulatively in the order phases first appeared. `undefined` means unset; `null` means set but never reached on the p50 path. */
45
+ /**
46
+ * Where a p50 run would be cut off by `max_run_tokens`, walking the phase
47
+ * breakdown cumulatively in the order phases first appeared. `undefined`
48
+ * means unset; `null` means set but never reached on the p50 path.
49
+ *
50
+ * Walks `billable_p50`, NOT `p50` — `defaults.max_run_tokens` is a BILLABLE
51
+ * ceiling (`assertRunBudget` in `core/agents.ts` checks it against
52
+ * `Run.billable_tokens`, never the display total), so comparing it against
53
+ * a cumulative TOTAL (cache reads included) would trip the projected cutoff
54
+ * far earlier than a real run ever would — the same total-vs-billable
55
+ * mismatch `UsageBreakdown.billable_tokens`'s doc comment (`core/data_types.ts`)
56
+ * describes for the real ceiling check itself.
57
+ */
42
58
  export declare function findCutoffPhase(phases: PhaseProjection[], maxRunTokens: number | undefined): string | null | undefined;
43
59
  export interface DriftWarning {
44
60
  agent: string;
@@ -95,19 +95,22 @@ export function normalizePhaseName(name) {
95
95
  return name.replace(/_\d+$/, "");
96
96
  }
97
97
  /**
98
- * Per run: sum every iteration of a normalized phase into one observation.
99
- * Across runs: median (p50) and min/max of those per-run sums. A phase
100
- * absent from a given run (a fix loop that never triggered) counts as a
101
- * `0` observation for that run — not a skip so its median reflects how
98
+ * Per run: sum every iteration of a normalized phase into one observation
99
+ * both the display total and the billable figure. Across runs: median (p50)
100
+ * and min/max of the total sums, plus the median of the billable sums. A
101
+ * phase absent from a given run (a fix loop that never triggered) counts as
102
+ * a `0` observation for that run — not a skip — so its median reflects how
102
103
  * often the phase actually ran, not just its size when it did.
103
104
  */
104
105
  export function aggregatePhases(sessions) {
105
106
  const perRunSums = sessions.map(() => new Map());
107
+ const perRunBillableSums = sessions.map(() => new Map());
106
108
  const firstSeq = new Map();
107
109
  sessions.forEach((session, i) => {
108
110
  for (const phase of session.phases) {
109
111
  const name = normalizePhaseName(phase.name);
110
112
  perRunSums[i].set(name, (perRunSums[i].get(name) ?? 0) + phase.tokens);
113
+ perRunBillableSums[i].set(name, (perRunBillableSums[i].get(name) ?? 0) + phase.billable_tokens);
111
114
  if (!firstSeq.has(name) || phase.seq < firstSeq.get(name))
112
115
  firstSeq.set(name, phase.seq);
113
116
  }
@@ -115,23 +118,36 @@ export function aggregatePhases(sessions) {
115
118
  const names = [...firstSeq.keys()].sort((a, b) => firstSeq.get(a) - firstSeq.get(b));
116
119
  return names.map((name) => {
117
120
  const perRun = perRunSums.map((m) => m.get(name) ?? 0);
118
- return { name, p50: median(perRun), min: Math.min(...perRun), max: Math.max(...perRun) };
121
+ const perRunBillable = perRunBillableSums.map((m) => m.get(name) ?? 0);
122
+ return { name, p50: median(perRun), min: Math.min(...perRun), max: Math.max(...perRun), billable_p50: median(perRunBillable) };
119
123
  });
120
124
  }
121
- /** The HEADLINE: median of sampled runs' `sessions.total_tokens` — NOT the sum of the per-phase medians (those are different numbers; median of sums != sum of medians). */
125
+ /** The HEADLINE: median of sampled runs' `sessions.total_tokens` — NOT the sum of the per-phase medians (those are different numbers; median of sums != sum of medians). This is the CONTEXT figure (cache reads included); it is never compared against `max_run_tokens` — see `findCutoffPhase` for the billable-based ceiling check. */
122
126
  export function projectRunTotals(sessions) {
123
127
  if (sessions.length === 0)
124
128
  return null;
125
129
  const totals = sessions.map((s) => s.total_tokens);
126
130
  return { p50: median(totals), min: Math.min(...totals), max: Math.max(...totals) };
127
131
  }
128
- /** Where a p50 run would be cut off by `max_run_tokens`, walking the phase breakdown cumulatively in the order phases first appeared. `undefined` means unset; `null` means set but never reached on the p50 path. */
132
+ /**
133
+ * Where a p50 run would be cut off by `max_run_tokens`, walking the phase
134
+ * breakdown cumulatively in the order phases first appeared. `undefined`
135
+ * means unset; `null` means set but never reached on the p50 path.
136
+ *
137
+ * Walks `billable_p50`, NOT `p50` — `defaults.max_run_tokens` is a BILLABLE
138
+ * ceiling (`assertRunBudget` in `core/agents.ts` checks it against
139
+ * `Run.billable_tokens`, never the display total), so comparing it against
140
+ * a cumulative TOTAL (cache reads included) would trip the projected cutoff
141
+ * far earlier than a real run ever would — the same total-vs-billable
142
+ * mismatch `UsageBreakdown.billable_tokens`'s doc comment (`core/data_types.ts`)
143
+ * describes for the real ceiling check itself.
144
+ */
129
145
  export function findCutoffPhase(phases, maxRunTokens) {
130
146
  if (maxRunTokens === undefined)
131
147
  return undefined;
132
148
  let cumulative = 0;
133
149
  for (const phase of phases) {
134
- cumulative += phase.p50;
150
+ cumulative += phase.billable_p50;
135
151
  if (cumulative >= maxRunTokens)
136
152
  return phase.name;
137
153
  }
@@ -289,6 +305,12 @@ function printText(report, coldStart) {
289
305
  for (const phase of report.phases) {
290
306
  lines.push(`${phase.name.padEnd(20)} ${fmt(phase.p50).padStart(12)} ${fmt(phase.min)} - ${fmt(phase.max)}`);
291
307
  }
308
+ // The display total above (cache reads included) is context only — it is
309
+ // never what `max_run_tokens` is checked against (see `findCutoffPhase`).
310
+ // Print the same billable sum that calc actually walks, so the ceiling
311
+ // line below (labeled "billable tokens") has a comparable figure on screen.
312
+ const billableTotal = report.phases.reduce((sum, p) => sum + p.billable_p50, 0);
313
+ lines.push(`billable p50 ${fmt(billableTotal).padStart(12)} (cache reads excluded; what max_run_tokens checks against)`);
292
314
  }
293
315
  if (report.projected) {
294
316
  lines.push("");
@@ -306,9 +328,9 @@ function printText(report, coldStart) {
306
328
  }
307
329
  lines.push("");
308
330
  if (report.ceilings.max_run_tokens !== undefined) {
309
- lines.push(`max_run_tokens ${fmt(report.ceilings.max_run_tokens)}` +
331
+ lines.push(`max_run_tokens ${fmt(report.ceilings.max_run_tokens)} (billable tokens — cache reads excluded)` +
310
332
  (report.ceilings.cutoff_phase ? ` -> would stop at "${report.ceilings.cutoff_phase}" on a p50 run` : ""));
311
- lines.push(' (checked BEFORE each call: a run can overshoot by one call, and a one-dispatch chain can never trip it)');
333
+ lines.push(' (checked BEFORE each call, against billable tokens: a run can overshoot by one call, and a one-dispatch chain can never trip it)');
312
334
  }
313
335
  if (report.ceilings.max_run_cost !== undefined) {
314
336
  lines.push(`max_run_cost $${report.ceilings.max_run_cost.toFixed(3)}`);
@@ -1,2 +1,22 @@
1
+ /**
2
+ * BILLABLE tokens for one ledger row's readback — `--max-tokens`
3
+ * (`overCumulativeBudget`, `core/loop.ts`) must compare against the same
4
+ * metric `defaults.max_run_tokens` checks per-call inside one chain run
5
+ * (see `LedgerAttempt.tokens`'s doc comment), never the display total
6
+ * (`total_tokens`, cache reads included).
7
+ *
8
+ * `billable_tokens` is 0 — never `null` — for BOTH a row that predates the
9
+ * column (`ALTER TABLE ... DEFAULT 0` backfills existing rows) and a
10
+ * genuinely free iteration, so there is no way to tell those apart from the
11
+ * value alone; falling back to `total_tokens` whenever `billable_tokens` is
12
+ * falsy is the pragmatic rule (a real dispatch's billable figure is
13
+ * essentially never exactly 0, so this only ever fires for a pre-tracking
14
+ * row or a free one, where 0 either way is the right answer). Exported so
15
+ * this exact rule is unit-tested directly, not just exercised end to end.
16
+ */
17
+ export declare function billableTokensFor(session: {
18
+ billable_tokens?: number | null;
19
+ total_tokens?: number | null;
20
+ } | null | undefined): number;
1
21
  export declare const USAGE: string;
2
22
  export declare function loopCommand(argv: string[]): Promise<number>;
@@ -8,6 +8,25 @@ import { SfDb } from "../../ui/server/db.js";
8
8
  import { parseCli, resolvePrompt } from "../../core/utils.js";
9
9
  import { isInteractive } from "../ask.js";
10
10
  import { resolveIssueProvider } from "./watch.js";
11
+ /**
12
+ * BILLABLE tokens for one ledger row's readback — `--max-tokens`
13
+ * (`overCumulativeBudget`, `core/loop.ts`) must compare against the same
14
+ * metric `defaults.max_run_tokens` checks per-call inside one chain run
15
+ * (see `LedgerAttempt.tokens`'s doc comment), never the display total
16
+ * (`total_tokens`, cache reads included).
17
+ *
18
+ * `billable_tokens` is 0 — never `null` — for BOTH a row that predates the
19
+ * column (`ALTER TABLE ... DEFAULT 0` backfills existing rows) and a
20
+ * genuinely free iteration, so there is no way to tell those apart from the
21
+ * value alone; falling back to `total_tokens` whenever `billable_tokens` is
22
+ * falsy is the pragmatic rule (a real dispatch's billable figure is
23
+ * essentially never exactly 0, so this only ever fires for a pre-tracking
24
+ * row or a free one, where 0 either way is the right answer). Exported so
25
+ * this exact rule is unit-tested directly, not just exercised end to end.
26
+ */
27
+ export function billableTokensFor(session) {
28
+ return session?.billable_tokens || session?.total_tokens || 0;
29
+ }
11
30
  /**
12
31
  * Just enough of `Run` for `quality.resolveSuite`/`runSuite` (`RunLike` in
13
32
  * `core/quality.ts`) to work against a real filesystem/console without a
@@ -152,7 +171,7 @@ export async function loopCommand(argv) {
152
171
  const db = await SfDb.open(dataPaths.db, dataPaths.sessions_dir);
153
172
  try {
154
173
  const session = await db.session(iteration.adw_id);
155
- tokens = session?.total_tokens ?? 0;
174
+ tokens = billableTokensFor(session);
156
175
  cost = session?.total_cost ?? 0;
157
176
  }
158
177
  finally {
@@ -4,7 +4,7 @@ import * as paths from "../../core/paths.js";
4
4
  import { parseCli } from "../../core/utils.js";
5
5
  import { runUi } from "../../ui/server/serve.js";
6
6
  export async function uiCommand(argv) {
7
- const { options, flags } = parseCli(argv, ["cwd", "config", "db", "port"], ["no-open"]);
7
+ const { options, flags } = parseCli(argv, ["cwd", "config", "db", "port", "host"], ["no-open"]);
8
8
  const anchor = paths.resolveAnchor(options["cwd"]);
9
9
  let db;
10
10
  let sessionsDir;
@@ -29,6 +29,7 @@ export async function uiCommand(argv) {
29
29
  sessionsDir,
30
30
  webDir: paths.WEB_DIR,
31
31
  port: options["port"] ? Number.parseInt(options["port"], 10) : undefined,
32
+ host: options["host"],
32
33
  open: !flags["no-open"],
33
34
  });
34
35
  console.log(`[spf] ui ${handle.url}`);
@@ -113,7 +113,7 @@ export function resolveIssueProvider(cfg) {
113
113
  console.error('JIRA_EMAIL and JIRA_API_TOKEN must both be set — spf watch needs an Atlassian account email plus an API token (id.atlassian.com -> Security -> API tokens). See README.md\'s "spf watch" section.');
114
114
  return null;
115
115
  }
116
- return new JiraProvider(cfg.watch.jira.base_url, cfg.watch.jira.project_key, cfg.watch.label_prefix, email, token, cfg.watch.jira.issue_types, cfg.watch.jira.status_map);
116
+ return new JiraProvider(cfg.watch.jira.base_url, cfg.watch.jira.project_key, cfg.watch.label_prefix, email, token, cfg.watch.jira.issue_types, cfg.watch.jira.status_map, cfg.watch.jira.link_type);
117
117
  }
118
118
  console.error(`watch.issue_provider ${JSON.stringify(cfg.watch.issue_provider)} is not supported`);
119
119
  return null;
package/dist/cli/index.js CHANGED
@@ -46,7 +46,7 @@ const HELP = `spf — repeatable agents-plus-code workflows (ADWs)
46
46
  spf migrate [--apply] [--force] move an old stamped adws/ tree onto .spf/ (dry run by default)
47
47
  spf eject [--target <dir>] [--force] copy the installed engine out for reference/hand-editing
48
48
  spf doctor [--json] check everything that fails silently otherwise
49
- spf ui [--port N] [--no-open] [--db path] open the trace visualizer
49
+ spf ui [--port N] [--host addr] [--no-open] [--db path] open the trace visualizer (loopback-only by default; --host widens it, deliberately)
50
50
  spf watch init idempotently seed the <prefix>:* labels watch.repo needs
51
51
  spf watch [--dry-run] [--once] poll watch.repo for labeled issues, run watch.chain on each
52
52
  spf sessions [--limit N] [--json] recent runs
@@ -56,7 +56,7 @@ const HELP = `spf — repeatable agents-plus-code workflows (ADWs)
56
56
  spf version print the installed version
57
57
 
58
58
  Chain options: [--config <path>] [--adw-id <id>] [--cwd <dir>] [--agent <name>] [--base <ref>] [--issue <id>] [--priority p0|p1|p2|p3]
59
- Run budget: set defaults.max_run_cost (USD) and/or defaults.max_run_tokens in spf.config.yaml to stop the NEXT agent call once a run has already spent this much — checked before each call, never after, so a single call is never capped and a one-agent-dispatch chain (scout/prompt/build) can never trip it; absent (the default) = unbounded.
59
+ Run budget: set defaults.max_run_cost (USD) and/or defaults.max_run_tokens in spf.config.yaml to stop the NEXT agent call once a run has already spent this much — checked before each call, never after, so a single call is never capped and a one-agent-dispatch chain (scout/prompt/build) can never trip it; absent (the default) = unbounded. max_run_tokens is checked against BILLABLE tokens (input + cache-write + output), never the larger display total shown elsewhere (cache reads included) — see the sessions panel's "tokens" line for that total instead. \`spf loop\`'s --max-tokens is the same billable metric, cumulative across every iteration.
60
60
  Run \`spf list\` to see every chain and what it needs.`;
61
61
  /** A raw scan for `--cwd`, ahead of any command-specific argv parsing — every command that takes it means the same thing by it. */
62
62
  function findCwdFlag(argv) {
@@ -265,6 +265,19 @@ export async function runInterview(asker, ctx) {
265
265
  env["OLLAMA_BASE_URL"] = baseUrl;
266
266
  envExampleKeys.push("OLLAMA_BASE_URL");
267
267
  asker.note("spf doctor checks this one — a probe against OLLAMA_BASE_URL/models runs on every `spf doctor`.");
268
+ // MINOR-F: optional per-gateway bearer — NOT added to
269
+ // PROVIDER_ENV_KEYS.ollama (see its doc comment: a bare local Ollama
270
+ // server needs no key at all, so this must stay optional, never
271
+ // required). A gateway in front of Ollama (e.g. Briefs' Envoy AI
272
+ // Gateway) does enforce a per-client key, though, which
273
+ // ollama_provider.ts's `ollamaApiKey()` sends as the bearer when
274
+ // set. Collected as an optional secret, same shape as the
275
+ // Cloudflare API token prompt below — blank is fine, left
276
+ // unanswered here just means the dummy placeholder keeps being used.
277
+ const ollamaApiKey = await asker.secret("OLLAMA_API_KEY", { current: ctx.existingEnv.get("OLLAMA_API_KEY") });
278
+ if (ollamaApiKey)
279
+ env["OLLAMA_API_KEY"] = ollamaApiKey;
280
+ envExampleKeys.push("OLLAMA_API_KEY");
268
281
  // Same problem the claude_code branch already solves above: the
269
282
  // packaged roster pins planner/reviewer/documenter to their own
270
283
  // fireworks/gemini/openai model strings, which always win over
@@ -51,16 +51,22 @@ function DashboardRoot(props) {
51
51
  setHistory((h) => [...h, { key, text }]);
52
52
  },
53
53
  setLivePhase: setLive,
54
- setUsage: (tokens, cost) => setUsage({ tokens, cost }),
54
+ setUsage: (tokens, cost, billableTokens) => setUsage({ tokens, cost, billableTokens }),
55
55
  };
56
56
  const overCost = props.maxCost !== undefined && usage.cost >= props.maxCost * 0.8;
57
- const overTokens = props.maxTokens !== undefined && usage.tokens >= props.maxTokens * 0.8;
58
- return (_jsxs(Box, { flexDirection: "column", children: [_jsx(Static, { items: history, children: (line) => _jsx(Text, { children: line.text }, line.key) }), live ? (_jsxs(Box, { children: [_jsx(Text, { color: "magenta", children: "\u25B8 running: " }), _jsx(Text, { bold: true, children: live.name }), _jsxs(Text, { dimColor: true, children: [" (", live.kind, " \u00B7 ", live.owner, ") "] }), _jsx(Elapsed, { sinceMs: live.startedAtMs })] })) : null, _jsxs(Box, { children: [_jsxs(Text, { dimColor: true, children: ["spend: ", usage.tokens.toLocaleString(), " tokens \u00B7 ", formatUsd(usage.cost), props.maxCost !== undefined ? ` / ${formatUsd(props.maxCost)}` : "", props.maxTokens !== undefined ? ` (of ${props.maxTokens.toLocaleString()} tokens)` : ""] }), overCost || overTokens ? _jsx(Text, { color: "yellow", children: " \u2014 approaching ceiling" }) : null] })] }));
57
+ // Compared against BILLABLE tokens, never the display total — `maxTokens`
58
+ // is `defaults.max_run_tokens`, which `assertRunBudget` (agents.ts) checks
59
+ // against `billable_tokens`, not `tokens` (cache reads included). Comparing
60
+ // the total here would trip "approaching ceiling" on cache-driven bulk
61
+ // that the real ceiling check never sees. See `UsageBreakdown.billable_tokens`'s
62
+ // doc comment (`data_types.ts`) for the full reasoning.
63
+ const overTokens = props.maxTokens !== undefined && usage.billableTokens >= props.maxTokens * 0.8;
64
+ return (_jsxs(Box, { flexDirection: "column", children: [_jsx(Static, { items: history, children: (line) => _jsx(Text, { children: line.text }, line.key) }), live ? (_jsxs(Box, { children: [_jsx(Text, { color: "magenta", children: "\u25B8 running: " }), _jsx(Text, { bold: true, children: live.name }), _jsxs(Text, { dimColor: true, children: [" (", live.kind, " \u00B7 ", live.owner, ") "] }), _jsx(Elapsed, { sinceMs: live.startedAtMs })] })) : null, _jsxs(Box, { children: [_jsxs(Text, { dimColor: true, children: ["spend: ", usage.billableTokens.toLocaleString(), " tokens \u00B7 ", formatUsd(usage.cost), props.maxCost !== undefined ? ` / ${formatUsd(props.maxCost)}` : "", props.maxTokens !== undefined ? ` (of ${props.maxTokens.toLocaleString()} tokens)` : "", ` · ${usage.tokens.toLocaleString()} tokens context incl. cache reads`] }), overCost || overTokens ? _jsx(Text, { color: "yellow", children: " \u2014 approaching ceiling" }) : null] })] }));
59
65
  }
60
66
  export function mountRunDashboard(opts) {
61
67
  const handleRef = { current: null };
62
68
  let currentPhase = null;
63
- let currentUsage = { tokens: 0, cost: 0 };
69
+ let currentUsage = { tokens: 0, cost: 0, billableTokens: 0 };
64
70
  // `undefined` while paused: Ink refuses a second `render()` on the same
65
71
  // stdout while a prior instance is still live (the same restriction
66
72
  // `ink_asker.tsx`'s confirm-timeout comment names), and the sign-off
@@ -104,10 +110,10 @@ export function mountRunDashboard(opts) {
104
110
  if (app)
105
111
  handleRef.current?.setLivePhase(null);
106
112
  },
107
- onUsage(tokens, cost) {
108
- currentUsage = { tokens, cost };
113
+ onUsage(tokens, cost, billableTokens) {
114
+ currentUsage = { tokens, cost, billableTokens };
109
115
  if (app)
110
- handleRef.current?.setUsage(tokens, cost);
116
+ handleRef.current?.setUsage(tokens, cost, billableTokens);
111
117
  },
112
118
  };
113
119
  async function unmount() {
@@ -115,17 +115,33 @@ export declare class CcToolCallTracker {
115
115
  private finish;
116
116
  }
117
117
  export declare function isKnownToolName(name: string): boolean;
118
+ /** This call's own SPF identity, when the caller has it — see `data_types.ts`'s `AgentRequest.adw_id`/`agent_name` doc. Both optional; an absent one simply omits its header line. */
119
+ export interface GatewayCallIdentity {
120
+ adwId?: string;
121
+ agentName?: string;
122
+ }
118
123
  /**
119
124
  * Merges `TRACEPARENT`/`ANTHROPIC_CUSTOM_HEADERS` into `baseEnv` for
120
- * outbound OTel propagation, or returns `baseEnv` UNCHANGED when
121
- * `otel` is absent (the common case — `observability.otel` not configured).
125
+ * outbound OTel + gateway-header propagation, or returns `baseEnv`
126
+ * UNCHANGED when NEITHER `otel` nor `gateway` has anything to contribute
127
+ * (no otel configured AND no adw_id/agent_name on this request — the
128
+ * byte-identical case for a bare, ungated `claude` install).
122
129
  * `ANTHROPIC_CUSTOM_HEADERS`'s verified format is newline-separated
123
130
  * `Name: Value` pairs (see this module's own header for the citation); an
124
131
  * operator-supplied value already present in `baseEnv` is kept and appended
125
132
  * to, not overwritten — a real header injected via config/settings must
126
133
  * still reach the wire alongside this module's own.
134
+ *
135
+ * `x-correlation-id`/`x-spf-agent` (from `gateway`) are added on EVERY call
136
+ * that has them, regardless of whether `otel` is configured — see
137
+ * `data_types.ts`'s `AgentRequest.adw_id` doc for why that pair is not
138
+ * gated on `observability.otel` the way `otel`/`traceparent` is. `traceparent`
139
+ * itself is added only when `otel` is present, unchanged from before.
140
+ * `x-request-id` is NEVER added — Envoy/Switchyard own that header
141
+ * end-to-end; this module used to send it here (BLOCKER B) and no longer
142
+ * does.
127
143
  */
128
- export declare function injectOtelEnv(baseEnv: Record<string, string>, otel: AgentRequest["otel"] | undefined): Record<string, string>;
144
+ export declare function injectOtelEnv(baseEnv: Record<string, string>, otel: AgentRequest["otel"] | undefined, gateway?: GatewayCallIdentity): Record<string, string>;
129
145
  /**
130
146
  * Resolve `SPF_CLAUDE_CMD` for THIS call, substituting a literal `{model}`
131
147
  * token with `model` — see the module doc comment for why a fixed tag baked
@@ -250,22 +250,40 @@ const EFFORT_MAP = {
250
250
  };
251
251
  /**
252
252
  * Merges `TRACEPARENT`/`ANTHROPIC_CUSTOM_HEADERS` into `baseEnv` for
253
- * outbound OTel propagation, or returns `baseEnv` UNCHANGED when
254
- * `otel` is absent (the common case — `observability.otel` not configured).
253
+ * outbound OTel + gateway-header propagation, or returns `baseEnv`
254
+ * UNCHANGED when NEITHER `otel` nor `gateway` has anything to contribute
255
+ * (no otel configured AND no adw_id/agent_name on this request — the
256
+ * byte-identical case for a bare, ungated `claude` install).
255
257
  * `ANTHROPIC_CUSTOM_HEADERS`'s verified format is newline-separated
256
258
  * `Name: Value` pairs (see this module's own header for the citation); an
257
259
  * operator-supplied value already present in `baseEnv` is kept and appended
258
260
  * to, not overwritten — a real header injected via config/settings must
259
261
  * still reach the wire alongside this module's own.
262
+ *
263
+ * `x-correlation-id`/`x-spf-agent` (from `gateway`) are added on EVERY call
264
+ * that has them, regardless of whether `otel` is configured — see
265
+ * `data_types.ts`'s `AgentRequest.adw_id` doc for why that pair is not
266
+ * gated on `observability.otel` the way `otel`/`traceparent` is. `traceparent`
267
+ * itself is added only when `otel` is present, unchanged from before.
268
+ * `x-request-id` is NEVER added — Envoy/Switchyard own that header
269
+ * end-to-end; this module used to send it here (BLOCKER B) and no longer
270
+ * does.
260
271
  */
261
- export function injectOtelEnv(baseEnv, otel) {
262
- if (!otel)
272
+ export function injectOtelEnv(baseEnv, otel, gateway = {}) {
273
+ const headerLines = [];
274
+ if (otel)
275
+ headerLines.push(`traceparent: ${otel.traceparent}`);
276
+ if (gateway.adwId)
277
+ headerLines.push(`x-correlation-id: ${gateway.adwId}`);
278
+ if (gateway.agentName)
279
+ headerLines.push(`x-spf-agent: ${gateway.agentName}`);
280
+ if (headerLines.length === 0)
263
281
  return baseEnv;
264
- const ownHeaders = [`traceparent: ${otel.traceparent}`, `x-request-id: ${otel.x_request_id}`].join("\n");
265
282
  const existing = baseEnv.ANTHROPIC_CUSTOM_HEADERS;
283
+ const ownHeaders = headerLines.join("\n");
266
284
  return {
267
285
  ...baseEnv,
268
- TRACEPARENT: otel.traceparent,
286
+ ...(otel ? { TRACEPARENT: otel.traceparent } : {}),
269
287
  ANTHROPIC_CUSTOM_HEADERS: existing ? `${existing}\n${ownHeaders}` : ownHeaders,
270
288
  };
271
289
  }
@@ -358,23 +376,25 @@ export async function run(request, onEvent, onSpawn, onExit) {
358
376
  const isOllamaLaunch = isOllamaLaunchCmd(cmdSpec);
359
377
  const needsOllamaLaunchSeparator = isOllamaLaunch && !cmdArgs.includes("--");
360
378
  const fullArgs = needsOllamaLaunchSeparator ? [...cmdArgs, "--", ...args] : [...cmdArgs, ...args];
361
- // Outbound OTel propagation (SPF's otel-sdk extension): `request.otel` is
362
- // set only when `observability.otel` is configured for this run (see
363
- // `agents.ts`'s `send()`) absent otherwise, so `env` below is
364
- // byte-identical to before this existed for every repo that hasn't
365
- // configured otel. `TRACEPARENT` is the standard W3C env var the `claude`
366
- // CLI's own subprocesses/telemetry already look for; `ANTHROPIC_CUSTOM_
367
- // HEADERS` additionally puts `traceparent` (redundant with the env var,
368
- // but this is the only way to reach the ACTUAL outbound HTTP request the
369
- // CLI itself makes to its configured `ANTHROPIC_BASE_URL`) and
370
- // `x-request-id` (this agent call's own span id) onto that request's
371
- // headers. Format verified against Claude Code's own docs (https://
379
+ // Outbound OTel + gateway-header propagation: `request.otel` is set only
380
+ // when `observability.otel` is configured for this run (see `agents.ts`'s
381
+ // `send()`); `request.adw_id`/`request.agent_name` are set on EVERY call
382
+ // regardless (see `data_types.ts`'s doc the gateway needs them whether
383
+ // or not SPF's own otel export is on). `env` below is byte-identical to
384
+ // before this existed for a request with neither. `TRACEPARENT` is the
385
+ // standard W3C env var the `claude` CLI's own subprocesses/telemetry
386
+ // already look for; `ANTHROPIC_CUSTOM_HEADERS` additionally puts
387
+ // `traceparent`/`x-correlation-id`/`x-spf-agent` onto the ACTUAL outbound
388
+ // HTTP request the CLI itself makes to its configured `ANTHROPIC_BASE_URL`
389
+ // NEVER `x-request-id` (Envoy/Switchyard own that header end-to-end;
390
+ // this module used to send it here — BLOCKER B — and no longer does).
391
+ // Format verified against Claude Code's own docs (https://
372
392
  // code.claude.com/docs/en/env-vars, fetched live for this feature —
373
393
  // requires CLI >= 2.1.227): "Custom headers to add to requests (`Name:
374
394
  // Value` format, newline-separated for multiple headers)". An
375
395
  // operator-supplied `ANTHROPIC_CUSTOM_HEADERS` already present in
376
396
  // `request.env` is PRESERVED, not clobbered — this appends to it.
377
- const env = injectOtelEnv(request.env ?? operatorEnv(), request.otel);
397
+ const env = injectOtelEnv(request.env ?? operatorEnv(), request.otel, { adwId: request.adw_id, agentName: request.agent_name });
378
398
  const child = spawn(cmd, fullArgs, { cwd: request.cwd, env });
379
399
  // The prompt travels as a positional argv element, not stdin — closing it
380
400
  // immediately avoids a real, observed ~3s "no stdin data received" stall