werkmeister-core 0.93.1 → 0.93.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,26 @@
1
1
  # sandbox-core
2
2
 
3
+ ## 0.93.3
4
+
5
+ ### Patch Changes
6
+
7
+ - c9fb647: SW-773: usage costs price the current models (Claude Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.2), so `usage.json` and the `workspace-usage` tool no longer show a null cost for every run. `~/.werkmeister/model-pricing.yaml` now overrides the built-in rates instead of replacing them, so a file seeded before a model shipped still prices it with no edit. A model with no rate is reported once instead of silently. The built-in and newly seeded rates for Claude Opus 4.8 and Sonnet 5 move to their published prices ($5/$25 and $2/$10). An existing `model-pricing.yaml` keeps its own older rows for those two models (15/75 and 3/15), and those still win. Delete those rows, or the whole file, to use the new rates.
8
+ - 058117c: SW-774: close-out's `usage.json` and `usage.md` no longer show negative stage minutes. A stage now ends at the earlier of the record-branch commit that first added a file to its `output/` (`.gitkeep` excluded) and its oldest output file, so stages committed together keep their own times. Before, it ended at its newest output file, so a later edit to an earlier stage made the next stage's duration negative.
9
+
10
+ ## 0.93.2
11
+
12
+ ### Patch Changes
13
+
14
+ - be37df0: Capacity queue (SW-682): a queued feature that can no longer fit the budget (queued before `budget.*` was lowered, or its compose grew) is dropped from the queue during a grant, and its session is told why with the NEVER_FITS message. The waiters behind it still start.
15
+ - b559753: Capacity queue (SW-686): containers a `--service` or gate start stopped while it waited for capacity now come back through `docker compose up --wait`, in dependency order and with health conditions, instead of one `docker start` per container by name. If they don't all come back, the error names the containers still stopped.
16
+ - 6db2de7: Capacity queue (SW-701): a `feature up --service` start that waits its 10 minutes without room now keeps its queue place instead of leaving it. It still exits 75, but it stays queued with its first wait time; when room frees, the grant pass starts that service, reloads the proxy and tells the session to continue. A rerun keeps its place. A parked start never holds the room against later starts, so a start that can't fit can't stall the queue. Gate starts, and starts that stopped their own containers to wait, don't park.
17
+ - 4667dd1: Capacity hook (SW-713): a session is no longer held by "Waiting for capacity" after its feature has left the queue. The hook clears a leftover waiting file instead of denying every tool call, and `feature restart` of a queued feature takes it out of the queue, so its running containers no longer leave the session stuck.
18
+ - 35638b3: Capacity queue (SW-737): a queued feature is started when room frees, even with no stop to trigger it. One waiter whose start failed (low memory or host load, a missing compose file) ended the whole grant pass, so every 10-minute `pr watch` tick failed on it and nothing behind it started, on an idle host too. The pass now skips that waiter and keeps it queued. A `--service` or gate start now runs a grant after it starts, and a start that gives up waiting runs one for those still queued.
19
+ - e417971: Capacity queue (SW-738): an agent's `feature up` in a person's workspace no longer starts as the person, so it can't stop another workspace's working containers. A start whose preemption fails to stop a container now puts that feature back in the queue to come back, and waits at the head of the queue itself, instead of leaving the other feature down and its own start not started.
20
+ - 6d6fb25: Capacity queue (SW-745): a running `pr gate` / `pr open` is never preempted. The gate marked itself busy only when it started a container or ran a row, so a test row longer than `idle_stop_minutes` read as not busy, and a queued start could stop its containers mid-run (exit 137, reported as test failures). The gate now refreshes that mark every minute while it runs. With `idle_stop_minutes: 0` a gate also counts as busy.
21
+ - c8669ea: Capacity queue (SW-763): the "waiting for capacity" and exit-75 messages now say what the start needs against what is free in the budget (`this start needs 4 CPUs, 1.0 GiB; free 3 CPUs, 5.8 GiB of the budget's 8 CPUs, 6.8 GiB`), so a request too big for the host reads apart from a busy one and can be retried with `--service`. The wake on an idle host comes from the `pr watch` grant pass, which SW-737 made work.
22
+ - b8d3c7b: Capacity hook (SW-768): while a workspace waits for capacity, its session is held only for commands that need containers or the CPUs (`docker`, `pnpm`/`npm`/`node` and other test runners, `with-temp-home`, `feature up`/`restart`, `pr gate`/`pr open`). Edits, reads, git on the record, `werkmeister feature stop` and prompts go through, so a person's approval can still be written and committed while queued. A paused workspace still holds everything.
23
+
3
24
  ## 0.93.1
4
25
 
5
26
  ### Patch Changes
@@ -111,6 +111,10 @@ export interface WaitingFile {
111
111
  gate?: true;
112
112
  /** ISO time of the first wait, kept across re-polls: the queue's starvation bound reads it (SW-675). */
113
113
  since?: string;
114
+ /** The `--service` start whose poll gave up but kept its place: the grant starts *service* (SW-701). */
115
+ parked?: {
116
+ service: string;
117
+ };
114
118
  }
115
119
  /** The waiting record, or null when there is none or it doesn't parse. */
116
120
  export declare function readWaiting(root: string): WaitingFile | null;
@@ -8,6 +8,7 @@
8
8
  */
9
9
  import * as fs from "node:fs";
10
10
  import * as path from "node:path";
11
+ import { sandboxHome } from "./config.js";
11
12
  import { dotDir } from "./dot_dir.js";
12
13
  import { WRAP_UP } from "./harness/claude/desktop.js";
13
14
  import { findRecordRoot } from "./pr_guard.js";
@@ -426,6 +427,7 @@ export function readWaiting(root) {
426
427
  ...(Array.isArray(o.services) ? { services: o.services.map(String) } : {}),
427
428
  ...(o.gate === true ? { gate: true } : {}),
428
429
  ...(typeof o.since === "string" ? { since: o.since } : {}),
430
+ ...(typeof o.parked?.service === "string" ? { parked: { service: o.parked.service } } : {}),
429
431
  };
430
432
  }
431
433
  catch {
@@ -438,7 +440,8 @@ export function liveWaiting(root) {
438
440
  // The poll rewrites the file every 15 s; a killed one leaves it behind (no finally on SIGTERM), and the
439
441
  // scheduler drops such a waiter after 60 s, so an older file is a dead wait, not a live one.
440
442
  const at = fs.statSync(waitingPath(root), { throwIfNoEntry: false })?.mtimeMs ?? 0;
441
- return waiting?.services && Date.now() - at > 60_000 ? null : waiting;
443
+ // A parked start has no poll by design (SW-701).
444
+ return waiting?.services && !waiting.parked && Date.now() - at > 60_000 ? null : waiting;
442
445
  }
443
446
  export function writeWaiting(root, w) {
444
447
  writeAtomic(waitingPath(root), w);
@@ -608,6 +611,49 @@ export function deliverWake(input, root) {
608
611
  ? { decision: "block", reason: message }
609
612
  : { hookSpecificOutput: { hookEventName: event, additionalContext: message } };
610
613
  }
614
+ /**
615
+ * A whole-feature waiting.json whose feature the queue no longer holds: a start or grant that dropped it
616
+ * without clearing the file, which held the session with a stale position (SW-713). Five seconds old at
617
+ * least, since admission writes this file before the queue (SW-246). Unsure (no FEATURE): not left over.
618
+ */
619
+ // scheduler_state's queuePath; not imported, since that module loads Docker and this hook runs on every tool call.
620
+ const queuePath = () => path.join(sandboxHome(), "scheduler", "queue.json");
621
+ function leftOver(root) {
622
+ const at = fs.statSync(waitingPath(root), { throwIfNoEntry: false })?.mtimeMs ?? Date.now();
623
+ if (Date.now() - at < 5_000)
624
+ return false;
625
+ let feature;
626
+ try {
627
+ feature = /^FEATURE=(.+)$/m.exec(fs.readFileSync(path.join(root, ".env"), "utf8"))?.[1]?.trim();
628
+ }
629
+ catch {
630
+ return false;
631
+ }
632
+ if (!feature)
633
+ return false;
634
+ try {
635
+ const queue = JSON.parse(fs.readFileSync(queuePath(), "utf8"));
636
+ return !(Array.isArray(queue) && queue.includes(feature));
637
+ }
638
+ catch {
639
+ return true; // no queue file: nothing waits
640
+ }
641
+ }
642
+ // Starts containers or runs tests. A list of what is held, not of what passes: the ticket names the held
643
+ // set, and an allowlist would hold `werkmeister feature stop`, the command that frees room (SW-768).
644
+ const NEEDS_CAPACITY = /(^|[\s;&|(`/'"])(docker|pnpm|npm|npx|node|turbo|vitest|with-temp-home|with-fixture-workspace)(\s|$)|\b(werkmeister|w9r)\s+(feature\s+(up|restart)|pr\s+(gate|open))\b/;
645
+ const NEEDS_CAPACITY_TOOL = /^mcp__.*werkmeister.*__(up|restart)$/;
646
+ function needsCapacity(input) {
647
+ if (input["hook_event_name"] !== "PreToolUse")
648
+ return false;
649
+ const tool = String(input["tool_name"] ?? "");
650
+ // A quoted message (`git commit -m "restart docker"`) is data, not a command (SW-768 review). Only
651
+ // `-m`/`--message`: other quotes may hold one (`sh -c 'docker …'`).
652
+ const command = String(input["tool_input"]?.["command"] ?? "").replace(/(\s(?:-m|--message)[= ]?\s*)('[^']*'|"[^"]*")/g, "$1''");
653
+ if (tool === "Bash")
654
+ return NEEDS_CAPACITY.test(command);
655
+ return NEEDS_CAPACITY_TOOL.test(tool);
656
+ }
611
657
  /**
612
658
  * The hook's stdout while the workspace is paused or waiting for capacity, or null. `continue: false` ends
613
659
  * the turn with the transcript intact; the deny also holds where a host ignores it.
@@ -622,8 +668,17 @@ export function pauseGate(input, root) {
622
668
  // and it must be able to run the tool that stops the wait (SW-545). Only a whole-feature queue holds it.
623
669
  if (waiting?.services)
624
670
  return null;
671
+ if (waiting && leftOver(root)) {
672
+ clearWaiting(root);
673
+ return null;
674
+ }
675
+ // Queued, only what needs this workspace's containers or the CPUs waits: the record, git and
676
+ // `feature stop` (which frees room) go on, and so do prompts (SW-768).
677
+ if (waiting && !needsCapacity(input))
678
+ return null;
625
679
  const reason = waiting
626
- ? `Waiting for capacity (position ${waiting.position}). End your turn now; it starts when another feature stops.`
680
+ ? `Waiting for capacity (position ${waiting.position}): containers and tests wait until this workspace starts, ` +
681
+ "and the session is told to continue then. Edits and git still work; end your turn if nothing else is left."
627
682
  : PAUSED_REASON;
628
683
  switch (input["hook_event_name"]) {
629
684
  case "PreToolUse": {
@@ -643,9 +698,7 @@ export function pauseGate(input, root) {
643
698
  // One guard for every sender: the person, claude-send, PR callbacks, check-in.
644
699
  return {
645
700
  decision: "block",
646
- reason: waiting
647
- ? "This workspace is waiting for capacity; it starts on its own."
648
- : "This workspace is paused. Press ▶ Resume on its card first.",
701
+ reason: "This workspace is paused. Press ▶ Resume on its card first.",
649
702
  };
650
703
  default:
651
704
  return null;
@@ -661,6 +714,14 @@ export function serviceWaitContext(input, root) {
661
714
  const waiting = liveWaiting(root);
662
715
  if (!waiting?.services)
663
716
  return null;
717
+ if (waiting.parked && leftOver(root)) {
718
+ clearWaiting(root); // its queue entry is gone (`feature down`, a dropped waiter): nothing will start it
719
+ return null;
720
+ }
721
+ if (waiting.parked)
722
+ return (`This workspace's start of ${waiting.parked.service} is queued for capacity (position ${waiting.position}). ` +
723
+ "Nothing polls for it: it starts by itself when room frees, and the session is told to continue. " +
724
+ "Rerunning `werkmeister feature up --service` keeps its place.");
664
725
  return (`This session's own service start is waiting for capacity (position ${waiting.position}): ` +
665
726
  `${waiting.services.join(", ")}. The werkmeister command that started it (pr gate, pr open or ` +
666
727
  "feature up --service), usually a background job in this session, polls every 15 s for up to " +
package/dist/compose.d.ts CHANGED
@@ -79,13 +79,14 @@ export declare function composeDown(feature: string, { removeVolumes }?: {
79
79
  /** Cancellable {@link composeUp}. */
80
80
  export declare function composeUpAsync(feature: string, signal?: AbortSignal): Promise<boolean>;
81
81
  /**
82
- * Start one service and wait until it runs (healthy, when it has a healthcheck).
82
+ * Start a service (or several, in compose's dependency order) and wait until each runs (healthy,
83
+ * when it has a healthcheck).
83
84
  *
84
85
  * Its setup service's `service_completed_successfully` holds the start until the
85
86
  * install finishes; *noDeps* skips that, for an install already current (SW-541).
86
87
  * An on-demand service starts without its profile enabled, because it is named (SW-34).
87
88
  */
88
- export declare function composeUpServiceAsync(feature: string, service: string, composePath: string, noDeps?: boolean): Promise<boolean>;
89
+ export declare function composeUpServiceAsync(feature: string, service: string | string[], composePath: string, noDeps?: boolean): Promise<boolean>;
89
90
  /** Cancellable {@link composeDown}. */
90
91
  export declare function composeDownAsync(feature: string, { removeVolumes, signal }?: {
91
92
  removeVolumes?: boolean;
package/dist/compose.js CHANGED
@@ -424,14 +424,15 @@ export async function composeUpAsync(feature, signal) {
424
424
  return false;
425
425
  }
426
426
  /**
427
- * Start one service and wait until it runs (healthy, when it has a healthcheck).
427
+ * Start a service (or several, in compose's dependency order) and wait until each runs (healthy,
428
+ * when it has a healthcheck).
428
429
  *
429
430
  * Its setup service's `service_completed_successfully` holds the start until the
430
431
  * install finishes; *noDeps* skips that, for an install already current (SW-541).
431
432
  * An on-demand service starts without its profile enabled, because it is named (SW-34).
432
433
  */
433
434
  export async function composeUpServiceAsync(feature, service, composePath, noDeps = false) {
434
- const args = ["up", "-d", "--wait", ...(noDeps ? ["--no-deps"] : []), service];
435
+ const args = ["up", "-d", "--wait", ...(noDeps ? ["--no-deps"] : []), ...[service].flat()];
435
436
  const result = await runComposeAsync(feature, args, {
436
437
  capture: true,
437
438
  composePath,
@@ -6,6 +6,7 @@ import fs from "node:fs/promises";
6
6
  import path from "node:path";
7
7
  import { parse as parseYaml } from "yaml";
8
8
  import { modelPricingYamlPath } from "../../config.js";
9
+ import { warn } from "../../reporter.js";
9
10
  import { isFile } from "../../tool_bins.js";
10
11
  import { findClaudeProjectDir, listSessionFiles, readSessionTitle } from "./transcript.js";
11
12
  const zeroTotals = () => ({ input: 0, output: 0, cacheCreation: 0, cacheRead: 0 });
@@ -87,10 +88,15 @@ export function rateFor(prices, model) {
87
88
  const base = model.replace(DATED_MODEL_ID_RE, "");
88
89
  return base !== model ? prices[base] : undefined;
89
90
  }
90
- // Fallback used when ~/.werkmeister/model-pricing.yaml is missing/unparseable.
91
+ // Under ~/.werkmeister/model-pricing.yaml: a file seeded before a model shipped still prices it (SW-773).
91
92
  const DEFAULT_PRICES = {
92
- "claude-opus-4-8": { input: 15, output: 75, cacheWrite: 18.75, cacheRead: 1.5 },
93
- "claude-sonnet-5": { input: 3, output: 15, cacheWrite: 3.75, cacheRead: 0.3 },
93
+ "claude-fable-5-2": { input: 8, output: 40, cacheWrite: 10, cacheRead: 0.2 },
94
+ "claude-opus-5-5": { input: 4, output: 20, cacheWrite: 5, cacheRead: 0.2 },
95
+ "claude-opus-4-8": { input: 5, output: 25, cacheWrite: 6.25, cacheRead: 0.5 },
96
+ "claude-sonnet-5-5": { input: 2, output: 10, cacheWrite: 2.5, cacheRead: 0.1 },
97
+ "claude-sonnet-5": { input: 2, output: 10, cacheWrite: 2.5, cacheRead: 0.2 },
98
+ // shortcut: ≤100K-token prompt tier only (over 100K costs 5×); add tiers if Haiku runs main sessions.
99
+ "claude-haiku-5-5": { input: 0.1, output: 0.5, cacheWrite: 0.125, cacheRead: 0.01 },
94
100
  "claude-haiku-4-5": { input: 1, output: 5, cacheWrite: 1.25, cacheRead: 0.1 },
95
101
  };
96
102
  /** Locate `~/.werkmeister/model-pricing.yaml`, mirroring resolveServicesYaml. */
@@ -104,7 +110,7 @@ export async function loadPriceTable() {
104
110
  return DEFAULT_PRICES;
105
111
  try {
106
112
  const doc = parseYaml(await fs.readFile(p, "utf-8"));
107
- return doc?.models && Object.keys(doc.models).length ? doc.models : DEFAULT_PRICES;
113
+ return { ...DEFAULT_PRICES, ...doc?.models };
108
114
  }
109
115
  catch {
110
116
  return DEFAULT_PRICES;
@@ -138,9 +144,20 @@ export async function usageForDir(name, target) {
138
144
  }
139
145
  return rollUpUsage(name, sessions);
140
146
  }
147
+ const warnedUnpriced = new Set();
141
148
  /** A workspace's sessions summed by model and by session, priced from the price table. */
142
149
  export async function rollUpUsage(name, tallies) {
143
150
  const prices = await loadPriceTable();
151
+ // An unpriced model left every cost null unnoticed for weeks (SW-773). Zero-token
152
+ // rows (Claude Code's `<synthetic>` error turns) cost nothing, so they don't warn.
153
+ for (const [model, t] of tallies.flatMap((s) => [...s.reduction.byModel])) {
154
+ if (!(t.input + t.output + t.cacheCreation + t.cacheRead))
155
+ continue;
156
+ if (rateFor(prices, model) || warnedUnpriced.has(model))
157
+ continue;
158
+ warnedUnpriced.add(model);
159
+ warn(`No price for model ${model}: its cost is unknown. Add it to ${modelPricingYamlPath()}.`);
160
+ }
144
161
  const wsByModel = new Map();
145
162
  const sessions = [];
146
163
  for (const { sessionId, title, updatedAt, reduction: session } of tallies) {
@@ -26,7 +26,7 @@
26
26
  import { type HostLookup } from "./feature_routing.js";
27
27
  import { type Note } from "./notes.js";
28
28
  import { type RunResult } from "./proc.js";
29
- import { type Lease, type Rank } from "./scheduler.js";
29
+ import { type Cost, type Lease, type Rank } from "./scheduler.js";
30
30
  import { type Starting } from "./scheduler_state.js";
31
31
  /**
32
32
  * The Claude layer's repair on every `up`. Each strategy repairs its own layer: jit
@@ -81,7 +81,7 @@ export interface UpResult {
81
81
  * `INSIDE_OWN_CONTAINER` before any of it when run inside *feature*'s own container.
82
82
  *
83
83
  * With no room in the budget it queues the feature instead and returns with
84
- * `waiting` set; *byPerson* (default: the workspace is not autonomous) may stop
84
+ * `waiting` set; *byPerson* (default: the workspace is not autonomous and no agent session runs it) may stop
85
85
  * working autonomous features to make room (SW-181); any start that doesn't fit
86
86
  * may stop features whose session is idle or waiting on a person (SW-546).
87
87
  */
@@ -104,8 +104,11 @@ export declare function up(feature: string, workspace: string, { signal, byPerso
104
104
  * Throws `INSIDE_OWN_CONTAINER`, `NO_COMPOSE`, `MISSING_SECRETS`, `SLOT_ALLOC`,
105
105
  * `NO_CAPACITY`, `NEVER_FITS`, `HOST_LOAD` or `COMPOSE_UP_FAILED`.
106
106
  */
107
- export declare function startService(feature: string, workspace: string, service: string, composeName?: string, { skipCurrentSetups }?: {
107
+ export declare function startService(feature: string, workspace: string, service: string, composeName?: string, { skipCurrentSetups, park, grant, waitMs, }?: {
108
108
  skipCurrentSetups?: boolean;
109
+ park?: boolean;
110
+ grant?: boolean;
111
+ waitMs?: number;
109
112
  }): Promise<void>;
110
113
  /**
111
114
  * Admit starting *service* (and what compose starts with it) against the budget,
@@ -115,10 +118,14 @@ export declare function startService(feature: string, workspace: string, service
115
118
  *
116
119
  * Throws `NO_CAPACITY` when the wait is spent, `NEVER_FITS` at once when the start costs more than the whole budget.
117
120
  */
118
- export declare function admitServiceStart(feature: string, workspace: string, service: string, { sleep, waitMs, noDeps, }?: {
121
+ export declare function admitServiceStart(feature: string, workspace: string, service: string, { sleep, waitMs, noDeps, park, grant, }?: {
119
122
  sleep?: (ms: number) => Promise<unknown>;
120
123
  waitMs?: number;
121
124
  noDeps?: boolean;
125
+ /** On timeout keep the queue place for a grant to start it (SW-701); never after a release. */
126
+ park?: boolean;
127
+ /** False inside a grant pass, which would otherwise wait on itself. */
128
+ grant?: boolean;
122
129
  }): Promise<void>;
123
130
  export declare const RANK_WORD: Record<Rank, string>;
124
131
  /** A running lease for a person to read: `name (3 CPUs, 0.8 GiB, idle, gate running)`. */
@@ -127,7 +134,7 @@ export declare function describeLease(l: Lease): string;
127
134
  * Who holds the budget, the head *feature* waits behind, and what happens next: the same lines for a
128
135
  * queued `up` and a timed-out service start (SW-675). *stays*: still queued, so the grant starts it.
129
136
  */
130
- export declare function capacityWait(feature: string, position: number, stays: boolean): string;
137
+ export declare function capacityWait(feature: string, position: number, stays: boolean, need: Cost): Promise<string>;
131
138
  /**
132
139
  * Reads the services a workspace runs: the `SERVICES` list in its `.env`, which
133
140
  * `werkmeister workspace init` writes. Runs in-process and only reads that file.
@@ -252,6 +259,11 @@ export interface RestartResult {
252
259
  * session is parked on a question, permission or plan prompt (SW-546); nor is the shared infra.
253
260
  */
254
261
  export declare function runningLeases(runCmd?: (cmd: string[]) => RunResult): Lease[] | null;
262
+ /**
263
+ * Whether a plain `up` here may stop working features: a person's workspace, not an agent in it. Claude
264
+ * Code sets CLAUDECODE=1 in its Bash tool, so an agent's start there admits as an agent (SW-738).
265
+ */
266
+ export declare function startsAsPerson(workspace: string, env?: NodeJS.ProcessEnv): boolean;
255
267
  /** Null when *feature* may start now (after any preemption); else its queue position. */
256
268
  export declare function admitOrQueue(feature: string, workspace: string, byPerson: boolean, starting?: Starting): Promise<number | null>;
257
269
  /**
package/dist/lifecycle.js CHANGED
@@ -26,7 +26,7 @@
26
26
  import * as fs from "node:fs";
27
27
  import * as path from "node:path";
28
28
  import { setTimeout as delay } from "node:timers/promises";
29
- import { agentOf, clearPaused, clearWaiting, nudgePath, readPaused, readSessions, readWaiting, STALE_SESSION_MS, writeNudge, writePaused, } from "./agent_state.js";
29
+ import { agentOf, clearPaused, clearWaiting, nudgePath, readPaused, readSessions, readWaiting, STALE_SESSION_MS, writeNudge, writePaused, writeWaiting, } from "./agent_state.js";
30
30
  import { ensureSharedNetwork, isCaddyRunning, reloadCaddy, startCaddy } from "./caddy.js";
31
31
  import { Colors } from "./colors.js";
32
32
  import {
@@ -52,7 +52,7 @@ import { info, raw, success, warn } from "./reporter.js";
52
52
  import { withStartRecord } from "./resources.js";
53
53
  import { queueOrder, STARVE_MS } from "./scheduler.js";
54
54
  import { schedulerOp, waitingQueue } from "./scheduler_client.js";
55
- import { dequeue, enqueue, featureCost, loadBudget, readLeases, readQueue, requestAdmission, serviceStarts, setupsCurrent, waiters, waitForSetupLoad, } from "./scheduler_state.js";
55
+ import { dequeue, enqueue, featureCost, loadBudget, readLeases, readQueue, requestAdmission, requeueFirst, serviceStarts, setupsCurrent, waiters, waitForSetupLoad, } from "./scheduler_state.js";
56
56
  import { getSecretsPath, loadSecrets, resolveSecrets } from "./secrets.js";
57
57
  import { getAllSlots, getPortRange, getServicePort, getSlot, LABEL_FEATURE, LEGACY_LABEL_FEATURE, releaseSlot, requireLeasedSlot, } from "./slots.js";
58
58
  import { strategyFor } from "./strategies/index.js";
@@ -94,7 +94,7 @@ export function refreshWorkspaceSettings(workspace) {
94
94
  * `INSIDE_OWN_CONTAINER` before any of it when run inside *feature*'s own container.
95
95
  *
96
96
  * With no room in the budget it queues the feature instead and returns with
97
- * `waiting` set; *byPerson* (default: the workspace is not autonomous) may stop
97
+ * `waiting` set; *byPerson* (default: the workspace is not autonomous and no agent session runs it) may stop
98
98
  * working autonomous features to make room (SW-181); any start that doesn't fit
99
99
  * may stop features whose session is idle or waiting on a person (SW-546).
100
100
  */
@@ -122,11 +122,11 @@ export async function up(feature, workspace, { signal, byPerson } = {}) {
122
122
  if (localServices.length) {
123
123
  // Before admission: a HOST_LOAD throw must not follow a preemption or a dequeue.
124
124
  await waitForSetupLoad(workspace, undefined);
125
- const waiting = await admitOrQueue(feature, workspace, byPerson ?? !readAutonomous(workspace));
125
+ const waiting = await admitOrQueue(feature, workspace, byPerson ?? startsAsPerson(workspace));
126
126
  if (waiting !== null) {
127
127
  warn(`Waiting for capacity, position ${waiting} — it starts when another feature stops`);
128
128
  return {
129
- waitingNote: capacityWait(feature, waiting, true),
129
+ waitingNote: await capacityWait(feature, waiting, true, featureCost(workspace)),
130
130
  feature,
131
131
  slot,
132
132
  portRange: [portStart, portEnd],
@@ -290,7 +290,7 @@ async function prepareSlot(feature, workspace, config, localServices) {
290
290
  * Throws `INSIDE_OWN_CONTAINER`, `NO_COMPOSE`, `MISSING_SECRETS`, `SLOT_ALLOC`,
291
291
  * `NO_CAPACITY`, `NEVER_FITS`, `HOST_LOAD` or `COMPOSE_UP_FAILED`.
292
292
  */
293
- export async function startService(feature, workspace, service, composeName = COMPOSE_FILE, { skipCurrentSetups = false } = {}) {
293
+ export async function startService(feature, workspace, service, composeName = COMPOSE_FILE, { skipCurrentSetups = false, park = false, grant = true, waitMs, } = {}) {
294
294
  refuseInsideOwnContainer(feature, "pr gate");
295
295
  migrateWorkspaceCompose(workspace);
296
296
  const composePath = path.join(workspace, COMPOSE_FILE);
@@ -306,9 +306,23 @@ export async function startService(feature, workspace, service, composeName = CO
306
306
  const noDeps = skipCurrentSetups && (await setupsCurrent(path.join(workspace, composeName), service));
307
307
  // Before admission, as in `up`: a HOST_LOAD throw must not follow a preemption.
308
308
  await waitForSetupLoad(workspace, noDeps ? [service] : serviceStarts(workspace, service));
309
- await admitServiceStart(feature, workspace, service, { noDeps });
310
- requireSharedNetwork();
311
- const started = await withStartRecord(workspace, "gate", () => composeUpServiceAsync(feature, service, path.join(workspace, composeName), noDeps));
309
+ await admitServiceStart(feature, workspace, service, {
310
+ noDeps,
311
+ park,
312
+ grant,
313
+ ...(waitMs !== undefined && { waitMs }),
314
+ });
315
+ let started;
316
+ try {
317
+ requireSharedNetwork();
318
+ started = await withStartRecord(workspace, "gate", () => composeUpServiceAsync(feature, service, path.join(workspace, composeName), noDeps));
319
+ }
320
+ finally {
321
+ // After the start, not at admission: admission prices only running containers, so a grant before
322
+ // it would hand a waiter the room this start is about to take (SW-737).
323
+ if (grant)
324
+ await grantIfQueued();
325
+ }
312
326
  if (!started) {
313
327
  // A timed-out or cancelled up leaves the container Created, never started (SW-86).
314
328
  const state = (await getContainerStatus(feature)).find((c) => c.service === service)?.status;
@@ -325,7 +339,7 @@ const SERVICE_WAIT_MS = 10 * 60_000;
325
339
  *
326
340
  * Throws `NO_CAPACITY` when the wait is spent, `NEVER_FITS` at once when the start costs more than the whole budget.
327
341
  */
328
- export async function admitServiceStart(feature, workspace, service, { sleep = delay, waitMs = SERVICE_WAIT_MS, noDeps = false, } = {}) {
342
+ export async function admitServiceStart(feature, workspace, service, { sleep = delay, waitMs = SERVICE_WAIT_MS, noDeps = false, park = false, grant = true, } = {}) {
329
343
  const ownRunning = (await getContainerStatus(feature)).filter((c) => c.status === "running");
330
344
  const up = new Set(ownRunning.map((c) => c.service));
331
345
  const services = (noDeps ? [service] : serviceStarts(workspace, service)).filter((s) => !up.has(s));
@@ -340,6 +354,7 @@ export async function admitServiceStart(feature, workspace, service, { sleep = d
340
354
  let starting = { services };
341
355
  let released = [];
342
356
  let admitted = false;
357
+ let parked = false;
343
358
  try {
344
359
  for (let waited = 0;; waited += SERVICE_POLL_MS) {
345
360
  // Never byPerson: an agent's gate in a person's workspace must not stop another run's containers.
@@ -356,19 +371,31 @@ export async function admitServiceStart(feature, workspace, service, { sleep = d
356
371
  }
357
372
  if (position === null) {
358
373
  admitted = true;
359
- for (const name of released)
360
- await docker().getContainer(name).start();
361
- if (waited)
362
- await grantWaitersQuietly(); // room left over may fit the next waiter
374
+ // Through compose, not by name: a db + app pair needs depends_on order and health waits (SW-686).
375
+ if (released.length &&
376
+ !(await composeUpServiceAsync(feature, [...up], path.join(workspace, COMPOSE_FILE), true))) {
377
+ const running = new Set((await getContainerStatus(feature))
378
+ .filter((c) => c.status === "running")
379
+ .map((c) => c.name));
380
+ throw new SandboxError(`Could not start ${[...up].join(", ")} again after waiting for capacity; still stopped: ` +
381
+ `${released.filter((n) => !running.has(n)).join(", ") || "none"}. ` +
382
+ "`werkmeister feature up` starts the feature again.", "COMPOSE_UP_FAILED");
383
+ }
363
384
  return;
364
385
  }
365
386
  if (waited >= waitMs) {
387
+ // A released set can't park: the grant would bring back *service*, not what it stopped.
388
+ const waiting = park && !released.length ? readWaiting(workspace) : null;
389
+ if (waiting) {
390
+ writeWaiting(workspace, { ...waiting, parked: { service } });
391
+ parked = true;
392
+ }
366
393
  throw new SandboxError(`No capacity to start ${services.join(", ")} for '${feature}' after ${waitMs / 60_000} min ` +
367
- `(queue position ${position}).` +
394
+ `(queue position ${position})${parked ? "; it stays queued" : ""}.` +
368
395
  (released.length
369
396
  ? ` ${[...up].join(", ")} stayed stopped; \`werkmeister feature up\` starts the feature again.`
370
397
  : "") +
371
- `\n${capacityWait(feature, position, false)}`, "NO_CAPACITY");
398
+ `\n${await capacityWait(feature, position, parked, featureCost(workspace, starting.services))}`, "NO_CAPACITY");
372
399
  }
373
400
  if (!waited) {
374
401
  warn(`Waiting for capacity to start ${services.join(", ")}: position ${position}, up to ${waitMs / 60_000} min`);
@@ -377,12 +404,20 @@ export async function admitServiceStart(feature, workspace, service, { sleep = d
377
404
  }
378
405
  }
379
406
  finally {
380
- if (!admitted) {
407
+ if (!admitted && !parked) {
381
408
  await dequeue(feature);
382
409
  clearWaiting(workspace);
383
410
  }
411
+ // Room it released, or held as the queue head, is free now and no stop will grant it (SW-737).
412
+ if (!admitted && grant)
413
+ await grantIfQueued();
384
414
  }
385
415
  }
416
+ /** A grant pass when anyone waits: room can free without a release, and only a grant starts a whole-feature waiter (SW-737). */
417
+ async function grantIfQueued() {
418
+ if (readQueue().length)
419
+ await grantWaitersQuietly();
420
+ }
386
421
  const gib = (b) => `${(b / 2 ** 30).toFixed(1)} GiB`;
387
422
  const cpus = (n) => `${Number(n.toFixed(1))} CPUs`;
388
423
  export const RANK_WORD = {
@@ -399,17 +434,25 @@ export function describeLease(l) {
399
434
  * Who holds the budget, the head *feature* waits behind, and what happens next: the same lines for a
400
435
  * queued `up` and a timed-out service start (SW-675). *stays*: still queued, so the grant starts it.
401
436
  */
402
- export function capacityWait(feature, position, stays) {
437
+ export async function capacityWait(feature, position, stays, need) {
403
438
  const running = runningLeases() ?? [];
404
439
  const lines = [
405
440
  `running: ${running.map(describeLease).join(", ") || "nothing werkmeister started"}`,
406
441
  ];
442
+ // What is asked against what is free, so a too-big request reads apart from a busy host (SW-763).
443
+ const budget = await loadBudget();
444
+ if (Number.isFinite(budget.cpus) && Number.isFinite(budget.memory)) {
445
+ const used = running.reduce((a, l) => ({ cpus: a.cpus + l.cost.cpus, memory: a.memory + l.cost.memory }), { cpus: 0, memory: 0 });
446
+ lines.push(`this start needs ${cpus(need.cpus)}, ${gib(need.memory)}; free ` +
447
+ `${cpus(Math.max(0, budget.cpus - used.cpus))}, ${gib(Math.max(0, budget.memory - used.memory))} ` +
448
+ `of the budget's ${cpus(budget.cpus)}, ${gib(budget.memory)}`);
449
+ }
407
450
  const head = position > 1 ? queueOrder(waiters(readQueue(), running))[0] : undefined;
408
451
  if (head && head.feature !== feature) {
409
452
  const ws = readLeases()[head.feature]?.workspace;
410
453
  const need = ws ? featureCost(ws, readWaiting(ws)?.services) : null;
411
454
  lines.push(`behind ${head.feature}${need ? ` (needs ${cpus(need.cpus)}, ${gib(need.memory)})` : ""}` +
412
- (head.waitedMs >= STARVE_MS
455
+ (head.waitedMs >= STARVE_MS && !head.parked // a parked head never holds (SW-701)
413
456
  ? `, which has waited ${Math.round(head.waitedMs / 60_000)} min and gets the next free room`
414
457
  : ""));
415
458
  }
@@ -696,7 +739,8 @@ export function runningLeases(runCmd) {
696
739
  const found = runningFeatures(runCmd);
697
740
  if (!found)
698
741
  return null;
699
- const gateCutoff = Date.now() - idleStopMinutes() * 60_000;
742
+ // At least two of the gate's 60 s heartbeats: idle_stop_minutes 0 (reaper off) made every gate preemptable (SW-745).
743
+ const gateCutoff = Date.now() - Math.max(idleStopMinutes(), 2) * 60_000;
700
744
  return found
701
745
  .map((f) => {
702
746
  const state = agentOf(readSessions(f.workspace));
@@ -722,6 +766,13 @@ export function runningLeases(runCmd) {
722
766
  .sort((a, b) => a.at - b.at)
723
767
  .map((x) => x.lease);
724
768
  }
769
+ /**
770
+ * Whether a plain `up` here may stop working features: a person's workspace, not an agent in it. Claude
771
+ * Code sets CLAUDECODE=1 in its Bash tool, so an agent's start there admits as an agent (SW-738).
772
+ */
773
+ export function startsAsPerson(workspace, env = process.env) {
774
+ return !readAutonomous(workspace) && !env["CLAUDECODE"];
775
+ }
725
776
  /** Null when *feature* may start now (after any preemption); else its queue position. */
726
777
  export function admitOrQueue(feature, workspace, byPerson, starting) {
727
778
  return schedulerOp("admit", { feature, workspace, byPerson, starting }, () => admitOrQueueHere(feature, workspace, byPerson, starting),
@@ -748,9 +799,19 @@ async function admitOrQueueHere(feature, workspace, byPerson, starting) {
748
799
  return decision.position;
749
800
  if (decision.kind === "preempt") {
750
801
  for (const victim of decision.victims) {
751
- // stop, not pause: a paused container keeps its memory.
752
- await stop(victim, { grant: false });
753
802
  const victimWs = readLeases()[victim]?.workspace || workspaceForFeature(victim);
803
+ try {
804
+ // stop, not pause: a paused container keeps its memory.
805
+ await stop(victim, { grant: false });
806
+ }
807
+ catch (e) {
808
+ // A failed stop can still have killed it (SW-738): queue it to come back, and wait rather than
809
+ // start over budget; this start had the head's turn, so it keeps it.
810
+ if (victimWs)
811
+ await enqueue(victim, victimWs);
812
+ warn(`Could not stop '${victim}' to make room (${e.message}); waiting instead`);
813
+ return requeueFirst(feature, workspace, starting);
814
+ }
754
815
  if (victimWs)
755
816
  await enqueue(victim, victimWs);
756
817
  const why = running.find((l) => l.feature === victim)?.rank;
@@ -790,10 +851,30 @@ async function grantWaitersNow() {
790
851
  await dequeue(feature);
791
852
  continue;
792
853
  }
854
+ const waiting = readWaiting(workspace);
855
+ if (waiting?.parked) {
856
+ if (await startParked(feature, workspace, waiting.parked.service)) {
857
+ started.push(feature);
858
+ if (liveSessions(workspace).length)
859
+ writeNudge(workspace, GRANTED_NUDGE);
860
+ }
861
+ continue;
862
+ }
793
863
  // A service start's own poll admits it; `up` would start the wrong set and dequeue it.
794
- if (readWaiting(workspace)?.services)
864
+ if (waiting?.services)
795
865
  continue;
796
- const r = await up(feature, workspace, { byPerson: false });
866
+ let r;
867
+ try {
868
+ r = await up(feature, workspace, { byPerson: false });
869
+ }
870
+ catch (e) {
871
+ // One waiter's failed re-up (HOST_LOAD, NO_COMPOSE, ...) must not end the pass for the rest (SW-737).
872
+ warn(`Could not start '${feature}' from the waiting queue: ${e.message}`);
873
+ // Admission already dropped it from the queue; its session is the only one that can act (SW-682).
874
+ if (e instanceof SandboxError && e.code === "NEVER_FITS")
875
+ writeNudge(workspace, e.message);
876
+ continue;
877
+ }
797
878
  // Backfill (SW-675): a later waiter may fit; admission itself holds the room for a starved head.
798
879
  if (r.waiting !== undefined)
799
880
  continue;
@@ -803,6 +884,31 @@ async function grantWaitersNow() {
803
884
  }
804
885
  return started;
805
886
  }
887
+ /**
888
+ * A parked `--service` start (SW-701), as `feature up --service` runs it, without its wait: no room
889
+ * parks it again in place. False when it did not start.
890
+ */
891
+ async function startParked(feature, workspace, service) {
892
+ try {
893
+ await startService(feature, workspace, service, COMPOSE_FILE, {
894
+ park: true,
895
+ waitMs: 0,
896
+ grant: false, // this is the grant pass: another would wait on it
897
+ });
898
+ }
899
+ catch (e) {
900
+ if (!(e instanceof SandboxError && e.code === "NO_CAPACITY")) {
901
+ const message = `Could not start '${feature}' ${service} from the waiting queue: ${e.message}`;
902
+ warn(message);
903
+ // Its session was told it continues once started; it must hear that it won't (SW-701 review).
904
+ writeNudge(workspace, message);
905
+ }
906
+ return false;
907
+ }
908
+ // startService skips Caddy, as `feature up --service` does before its own reload (SW-338).
909
+ await reloadProxy().catch((e) => warn(`Proxy not reloaded for '${feature}': ${e.message}`));
910
+ return true;
911
+ }
806
912
  /**
807
913
  * After a release: a client hands the queue to the daemon and returns; the
808
914
  * daemon, or a client without one, starts waiters here. Never fails the stop
@@ -859,6 +965,10 @@ export async function restart(feature) {
859
965
  if (!restarted) {
860
966
  throw new SandboxError(`Failed to restart containers for '${feature}'`, "COMPOSE_RESTART_FAILED");
861
967
  }
968
+ // Running again, so no longer waiting: a queue entry left here held its session over live containers (SW-713).
969
+ await dequeue(feature);
970
+ if (workspace)
971
+ clearWaiting(workspace);
862
972
  return { feature, slot };
863
973
  }
864
974
  /** Containers running while the session is held would undo the pause: Resume is the way back. */
@@ -128,6 +128,8 @@ async function stopWorkspace(ws) {
128
128
  return `could not stop containers — ${e.message}`;
129
129
  }
130
130
  }
131
+ /** Well inside the shortest sane idle_stop_minutes, so a running gate always reads as busy. */
132
+ const EXEC_BEAT_MS = 60_000;
131
133
  /** Record that the gate is about to `docker exec`, so the idle reaper keeps the containers. */
132
134
  function touchExecMarker(ws) {
133
135
  try {
@@ -486,10 +488,15 @@ export async function runGate(ws, file, deps = defaultDeps(), { testsOnly = fals
486
488
  const started = [];
487
489
  const recreated = [];
488
490
  let result;
491
+ // A row can outlast idle_stop_minutes; the scheduler then read the gate as not busy and a queue
492
+ // head preempted its containers mid-run (SW-745, SW-738).
493
+ const beat = setInterval(() => touchExecMarker(ws), EXEC_BEAT_MS);
494
+ beat.unref();
489
495
  try {
490
496
  result = await gateChecks(ws, file, deps, started, recreated, testsOnly);
491
497
  }
492
498
  finally {
499
+ clearInterval(beat);
493
500
  fs.rmSync(path.join(ws, GATE_COMPOSE_FILE), { force: true });
494
501
  const feature = String(loadConfigFile(ws, null)["feature"] ?? readEnv(ws, "FEATURE")).trim();
495
502
  for (const svc of recreated) {
@@ -20,7 +20,7 @@ export interface SeedResult {
20
20
  status: "created" | "present";
21
21
  }
22
22
  export declare const DEFAULT_PORTS_YAML = "# Port Allocation Scheme\n# Each feature slot (1-12) gets a dedicated port range.\n#\n# k8s port-forward listen-ports (the 9000-range) are NOT defined here \u2014 they\n# live per-service in services.yaml as `k8s_forward_port:` (single source of\n# truth for the forward table).\n\nfeature_slots:\n count: 12\n base_port: 11100\n ports_per_slot: 20\n # Slot 1: 11100-11119\n # Slot 2: 11200-11219\n # ...\n # Slot 12: 12200-12219\n";
23
- export declare const DEFAULT_MODEL_PRICING_YAML = "# Per-million-token USD rates for estimating Claude Code usage cost.\n# Add or reprice models here \u2014 no code change needed. Unknown models show\n# token counts with no dollar estimate. Rates are approximate; update when\n# published pricing changes.\nmodels:\n claude-opus-4-8: { input: 15, output: 75, cacheWrite: 18.75, cacheRead: 1.50 }\n claude-sonnet-5: { input: 3, output: 15, cacheWrite: 3.75, cacheRead: 0.30 }\n claude-haiku-4-5: { input: 1, output: 5, cacheWrite: 1.25, cacheRead: 0.10 }\n";
23
+ export declare const DEFAULT_MODEL_PRICING_YAML = "# Per-million-token USD rates for estimating Claude Code usage cost.\n# Add or reprice models here \u2014 no code change needed. A rate here overrides\n# werkmeister's built-in one; a model missing here falls back to the built-in\n# rate. Unknown models show token counts with no dollar estimate and a warning.\n# Rates are approximate; update when published pricing changes.\nmodels:\n claude-fable-5-2: { input: 8, output: 40, cacheWrite: 10, cacheRead: 0.20 }\n claude-opus-5-5: { input: 4, output: 20, cacheWrite: 5, cacheRead: 0.20 }\n claude-opus-4-8: { input: 5, output: 25, cacheWrite: 6.25, cacheRead: 0.50 }\n claude-sonnet-5-5: { input: 2, output: 10, cacheWrite: 2.50, cacheRead: 0.10 }\n claude-sonnet-5: { input: 2, output: 10, cacheWrite: 2.50, cacheRead: 0.20 }\n claude-haiku-5-5: { input: 0.10, output: 0.50, cacheWrite: 0.125, cacheRead: 0.01 } # \u2264100K-token prompts\n claude-haiku-4-5: { input: 1, output: 5, cacheWrite: 1.25, cacheRead: 0.10 }\n";
24
24
  /**
25
25
  * Render the starter secrets.env template.
26
26
  *
package/dist/scaffold.js CHANGED
@@ -38,13 +38,18 @@ feature_slots:
38
38
  // model-pricing.yaml
39
39
  // ---------------------------------------------------------------------------
40
40
  export const DEFAULT_MODEL_PRICING_YAML = `# Per-million-token USD rates for estimating Claude Code usage cost.
41
- # Add or reprice models here — no code change needed. Unknown models show
42
- # token counts with no dollar estimate. Rates are approximate; update when
43
- # published pricing changes.
41
+ # Add or reprice models here — no code change needed. A rate here overrides
42
+ # werkmeister's built-in one; a model missing here falls back to the built-in
43
+ # rate. Unknown models show token counts with no dollar estimate and a warning.
44
+ # Rates are approximate; update when published pricing changes.
44
45
  models:
45
- claude-opus-4-8: { input: 15, output: 75, cacheWrite: 18.75, cacheRead: 1.50 }
46
- claude-sonnet-5: { input: 3, output: 15, cacheWrite: 3.75, cacheRead: 0.30 }
47
- claude-haiku-4-5: { input: 1, output: 5, cacheWrite: 1.25, cacheRead: 0.10 }
46
+ claude-fable-5-2: { input: 8, output: 40, cacheWrite: 10, cacheRead: 0.20 }
47
+ claude-opus-5-5: { input: 4, output: 20, cacheWrite: 5, cacheRead: 0.20 }
48
+ claude-opus-4-8: { input: 5, output: 25, cacheWrite: 6.25, cacheRead: 0.50 }
49
+ claude-sonnet-5-5: { input: 2, output: 10, cacheWrite: 2.50, cacheRead: 0.10 }
50
+ claude-sonnet-5: { input: 2, output: 10, cacheWrite: 2.50, cacheRead: 0.20 }
51
+ claude-haiku-5-5: { input: 0.10, output: 0.50, cacheWrite: 0.125, cacheRead: 0.01 } # ≤100K-token prompts
52
+ claude-haiku-4-5: { input: 1, output: 5, cacheWrite: 1.25, cacheRead: 0.10 }
48
53
  `;
49
54
  // ---------------------------------------------------------------------------
50
55
  // secrets.env
@@ -40,6 +40,8 @@ export interface Waiter {
40
40
  gate: boolean;
41
41
  /** A whole-feature waiter's cost; unset for a services start, whose own poll admits it. */
42
42
  cost?: Cost;
43
+ /** A `--service` start that gave up polling and waits for a grant (SW-701): it never holds the room. */
44
+ parked?: boolean;
43
45
  }
44
46
  /** How long the queue head waits before backfill stops passing it and a plain `up` head may preempt (SW-675). */
45
47
  export declare const STARVE_MS: number;
package/dist/scheduler.js CHANGED
@@ -60,7 +60,12 @@ export function admit(budget, running, queue, req) {
60
60
  const headAwaitsGrant = !!headCost &&
61
61
  used.memory + headCost.memory <= budget.memory &&
62
62
  used.cpus + headCost.cpus <= budget.cpus;
63
- const headHolds = !isHead && !endsGate && !headAwaitsGrant && (order[0].gate || order[0].waitedMs >= STARVE_MS);
63
+ // A parked head never holds: it may wait without end, and a hold would then hold for good (SW-701).
64
+ const headHolds = !isHead &&
65
+ !endsGate &&
66
+ !headAwaitsGrant &&
67
+ !order[0].parked &&
68
+ (order[0].gate || order[0].waitedMs >= STARVE_MS);
64
69
  if (fits({ memory: 0, cpus: 0 }) && (req.byPerson || !headHolds))
65
70
  return { kind: "admit" };
66
71
  // An idle or human-check lease (a session parked on a question) has nobody in it, so the queue head
@@ -27,6 +27,8 @@ export declare function readQueue(): string[];
27
27
  export declare function dequeue(feature: string): Promise<void>;
28
28
  /** Queue *feature* at the back and hold its session (a preempted feature). */
29
29
  export declare function enqueue(feature: string, workspace: string): Promise<void>;
30
+ /** Put *feature* back at the head as a capacity waiter; returns its position, 1. */
31
+ export declare function requeueFirst(feature: string, workspace: string, starting?: Starting): Promise<number>;
30
32
  /**
31
33
  * The budget: `budget:` in `~/.werkmeister/config.yaml` over defaults from the
32
34
  * engine (memory minus 1G for the VM itself, every CPU). An engine that can't
@@ -84,6 +84,19 @@ export function enqueue(feature, workspace) {
84
84
  writeQueue([...queue, feature]);
85
85
  }, { timeout: LOCK_TIMEOUT });
86
86
  }
87
+ /** Put *feature* back at the head as a capacity waiter; returns its position, 1. */
88
+ export function requeueFirst(feature, workspace, starting) {
89
+ return sandboxLock("scheduler", async () => {
90
+ writeWaiting(workspace, {
91
+ position: 1,
92
+ reason: "waiting for capacity",
93
+ ...starting,
94
+ since: new Date().toISOString(),
95
+ });
96
+ writeQueue([feature, ...readQueue().filter((f) => f !== feature)]);
97
+ return 1;
98
+ }, { timeout: LOCK_TIMEOUT });
99
+ }
87
100
  /**
88
101
  * The budget: `budget:` in `~/.werkmeister/config.yaml` over defaults from the
89
102
  * engine (memory minus 1G for the VM itself, every CPU). An engine that can't
@@ -341,8 +354,9 @@ function withoutDeadWaiters(queue) {
341
354
  const leases = readLeases();
342
355
  return queue.filter((f) => {
343
356
  const ws = leases[f]?.workspace;
344
- if (!ws || !readWaiting(ws)?.services)
345
- return true;
357
+ const w = ws ? readWaiting(ws) : null;
358
+ if (!ws || !w?.services || w.parked)
359
+ return true; // a parked start has no poll by design (SW-701)
346
360
  const at = fs.statSync(waitingPath(ws), { throwIfNoEntry: false })?.mtimeMs ?? 0;
347
361
  if (Date.now() - at < WAITER_STALE_MS)
348
362
  return true;
@@ -357,6 +371,16 @@ export function waiters(queue, running, now = Date.now()) {
357
371
  const ws = leases[feature]?.workspace;
358
372
  const w = ws ? readWaiting(ws) : null;
359
373
  const since = w?.since ? Date.parse(w.since) : Number.NaN;
374
+ if (ws && w?.parked) {
375
+ // Started only by a grant, like a whole-feature waiter, so priced like one; never a gate (SW-701).
376
+ return {
377
+ feature,
378
+ waitedMs: Number.isFinite(since) ? Math.max(0, now - since) : 0,
379
+ gate: false,
380
+ parked: true,
381
+ cost: featureCost(ws, w.services),
382
+ };
383
+ }
360
384
  return {
361
385
  feature,
362
386
  waitedMs: Number.isFinite(since) ? Math.max(0, now - since) : 0,
@@ -29,7 +29,11 @@ export interface UsageSnapshot {
29
29
  }
30
30
  export declare function formatTokens(n: number): string;
31
31
  export declare function usageSummary(s: UsageSnapshot): string;
32
- /** Each `stages/NN_*` finished at its newest output file; mtimes, so approximate. */
32
+ /**
33
+ * Each `stages/NN_*` finished at the earlier of its first output commit on the record branch and
34
+ * its oldest output file's mtime. Never the newest: re-editing an earlier stage's file later made
35
+ * the next stage's minutes negative (SW-774).
36
+ */
33
37
  export declare function stageTimings(ws: string, firstTurn: string | null): StageTiming[];
34
38
  export declare function buildUsageSnapshot(ws: string, name: string, cfg: Record<string, unknown>, outcome: UsageSnapshot["outcome"], closedAt?: string): Promise<UsageSnapshot>;
35
39
  export declare function renderUsageMd(s: UsageSnapshot): string;
@@ -3,6 +3,7 @@
3
3
  * `usage.md` for people. The transcripts it reads are keyed by the workspace path, so once the
4
4
  * folder is removed these files are the only record of what the run cost.
5
5
  */
6
+ import { spawnSync } from "node:child_process";
6
7
  import * as fs from "node:fs";
7
8
  import * as path from "node:path";
8
9
  import { sandboxRoot } from "./config.js";
@@ -32,7 +33,11 @@ export function usageSummary(s) {
32
33
  const cost = s.estUsd == null ? "cost unknown" : `~${usd(s.estUsd)}`;
33
34
  return `Usage: ${formatTokens(tokensOf(s.totals))} tokens, ${cost} across ${s.sessionCount} session${s.sessionCount === 1 ? "" : "s"}`;
34
35
  }
35
- /** Each `stages/NN_*` finished at its newest output file; mtimes, so approximate. */
36
+ /**
37
+ * Each `stages/NN_*` finished at the earlier of its first output commit on the record branch and
38
+ * its oldest output file's mtime. Never the newest: re-editing an earlier stage's file later made
39
+ * the next stage's minutes negative (SW-774).
40
+ */
36
41
  export function stageTimings(ws, firstTurn) {
37
42
  const root = path.join(ws, "stages");
38
43
  let dirs;
@@ -46,8 +51,8 @@ export function stageTimings(ws, firstTurn) {
46
51
  catch {
47
52
  return [];
48
53
  }
49
- const newest = (dir) => {
50
- let max = 0;
54
+ const oldest = (dir) => {
55
+ let min = 0;
51
56
  let entries;
52
57
  try {
53
58
  entries = fs.readdirSync(dir, { withFileTypes: true });
@@ -57,16 +62,23 @@ export function stageTimings(ws, firstTurn) {
57
62
  }
58
63
  for (const e of entries) {
59
64
  const p = path.join(dir, e.name);
60
- if (e.isDirectory())
61
- max = Math.max(max, newest(p));
62
- else if (e.name !== ".gitkeep")
63
- max = Math.max(max, fs.statSync(p).mtimeMs);
65
+ const t = e.isDirectory() ? oldest(p) : e.name === ".gitkeep" ? 0 : fs.statSync(p).mtimeMs;
66
+ if (t && (!min || t < min))
67
+ min = t;
64
68
  }
65
- return max;
69
+ return min;
70
+ };
71
+ const firstCommitted = (stage) => {
72
+ const out = `stages/${stage}/output`;
73
+ const r = spawnSync("git", ["-C", ws, "log", "--diff-filter=A", "--format=%cI", "--", out, `:!${out}/.gitkeep`], { encoding: "utf8" });
74
+ const first = r.status === 0 ? r.stdout.trim().split("\n").pop() : ""; // newest first
75
+ return first ? Date.parse(first) : 0;
66
76
  };
67
77
  let prev = firstTurn;
68
78
  return dirs.map((stage) => {
69
- const ms = newest(path.join(root, stage, "output"));
79
+ // The earlier of the two: a commit can bundle several stages, and a checkout resets mtimes.
80
+ const times = [firstCommitted(stage), oldest(path.join(root, stage, "output"))].filter(Boolean);
81
+ const ms = times.length ? Math.min(...times) : 0;
70
82
  const finishedAt = ms ? new Date(ms).toISOString() : null;
71
83
  const minutes = finishedAt ? minutesBetween(prev, finishedAt) : null;
72
84
  if (finishedAt)
@@ -173,7 +185,7 @@ export function renderUsageMd(s) {
173
185
  "",
174
186
  ...table("| session | title | first turn | last turn | turns | tokens | model(s) | est. USD |\n|---|---|---|---|---|---|---|---|", s.sessions.map((x) => `| ${x.sessionId} | ${cell(x.title)} | ${cell(x.firstTurn)} | ${cell(x.lastTurn)} | ${x.turns} | ${formatTokens(tokensOf(x.totals))} | ${cell(x.models.join(", "))} | ${usd(x.estUsd)} |`)),
175
187
  "",
176
- "## Stages (approximate: from output file times)",
188
+ "## Stages (from each stage's first output commit or oldest output file, whichever is earlier)",
177
189
  "",
178
190
  ...table("| stage | finished | minutes |\n|---|---|---|", s.stages.map((t) => `| ${t.stage} | ${t.finishedAt ?? "not reached"} | ${cell(t.minutes)} |`)),
179
191
  "",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "werkmeister-core",
3
- "version": "0.93.1",
3
+ "version": "0.93.3",
4
4
  "license": "Apache-2.0",
5
5
  "type": "module",
6
6
  "description": "werkmeister's shared library. Its public API is werkmeister-core/sdk; every other entry point is internal.",