tldr-experts 0.31.1 → 0.32.0

package/CHANGELOG.md

@@ -1,5 +1,180 @@
 # Changelog
 
+## 0.32.0 — 2026-09-16
+
+### Fixed
+
+- **A refused COMPOUND ad hoc Bash line — never the Definition of Done command itself — was
+  turning an ordinary red DoD into a human `blocked` (#360).** Measured on a live unattended run:
+  4 of 4 refusals were the developer's own ad hoc verification line (a `dotnet format … > log
+  2>&1; echo …`, a `;`-chained `find`, a piped `find … | xargs cat`, a `docker info … && echo …`
+  — none of them the DoD), and 2 of 5 stories blocked at attempt 1 of 2 even though each had
+  committed work and a DoD that went on to run and decide, each costing an operator `story
+  reopen` + `reject --and-continue` (22–27 minutes measured twice). The #271/#313 rule reasoned
+  that "the same allowance would refuse the same command again", which is true of a verbatim
+  re-ask but not of a compound line refused for its SHAPE: `dodRedRequeue`
+  (`src/core/build/reviewRound.ts`) now requeues a red DoD, like any other, when the refusal
+  `classifyRefusal` (`src/core/build/refusalKind.ts`) reads as `separator` (chained — `&&`, `;`,
+  a pipe, a redirect, a heredoc) or `undeclared` (a command no `commands:` slot grants) — an
+  ungranted git `verb` or `git -C` (`elsewhere`) refusal still blocks on the first attempt,
+  because THAT line, verbatim, would be refused again. The refusal stays on the record either
+  way; only the block is narrowed.
+- **The developer prompt told every Bash call to run one command alone only for the Definition of
+  Done (#360).** `## Rules` (`src/core/build/prompts.ts`) carried the "no redirection, pipes or
+  chaining" sentence beside the DoD rule alone, so a developer choosing its OWN ad hoc
+  verification command — 4 of 4 refusals in the run above — never read it. The rule now covers
+  every Bash call: "one command per Bash call, run verbatim and alone … no redirection, pipes,
+  chaining or heredocs … a compound line is refused unread". This changes the golden developer
+  prompt's bytes on purpose — `test/build-golden.test.ts`'s fixtures are updated for exactly
+  those lines.
+- **A story released from a stale dependency hold could sit `blocked` forever if the gh #298
+  rate-limit park landed on its own turn (#361).** `staleDependencyHold` deciding a `blocked`
+  row's dependency is now `done` only added the story to this pass's `pending` list and logged a
+  line — the story file itself stayed `blocked` until `driveStory` actually settled the attempt,
+  and `driveStory`'s very first act is the rate-limit park, which returns before any write when
+  the wall is already up from an earlier story's turn. Measured on a live unattended run
+  (0.31.1): a story sat `blocked` across a whole further invocation, with `gate.requested`
+  reporting the generic `settled by an earlier \`tldrx next\`` reason a never-touched row falls
+  back to, and an operator had to `story reopen` it by hand. The release is now written to the
+  story file (`blocked` → `todo`) the moment it is decided, so a park before the attempt leaves
+  the story exactly where an ordinary un-attempted `todo` row sits — picked up plainly by the
+  next invocation instead of re-deriving the same release from the log every time it gets that
+  far.
+- **An oversized `permission_refused` failed the whole BUILD STAGE, not just the story, and bought
+  `run auto` a relaunch nobody owed (#359).** Measured on a live unattended run (0.31.1): a
+  developer's refused command was a ~110-line heredoc; `permission_refused` (#271, additive) is
+  copied verbatim and was never on the emit seam's droppable-field list
+  (`detail`/`recording_error`/`outputs`/the `touches_widened` path lists), so a 5474-byte refused
+  command alone put a real `task.done` over the §2.9 4096-byte cap, `EventLog.append` refused it,
+  the executor threw, and the stage failed — re-dispatching a sibling story's developer for a turn
+  its own attempt never spent. `permission_refused` and `budget_death` (#277, the same shape) are
+  now clamped in `executors/build.ts`'s `settle` — head, bounded to 1024 bytes (a quarter of the
+  cap, `dodOutput.ts`'s existing budget, reused rather than re-derived), plus a marker naming the
+  original byte count and pointing at `04-build/log/<story>.md`, which already carries the field
+  in full (written by `writeLog` before this event, so the pointer is true by construction). The
+  review log, the handoff and the blocked-story reason are unaffected — none of them carry the
+  event-payload cap.
+- **A rate-limit `allowed_warning` parked a Build stage's whole dispatch at ANY utilization,
+  turning an unattended run into a human gate seven times over 53–60% of the seven-day window
+  (#367).** `noteRateLimit` (`src/core/facilitator/executors/build.ts`) now parks an
+  `allowed_warning` only at or above 90% utilization of the window that carries it (owner
+  decision, Slack, 2026-09-16); below the line the warning is recorded on the same
+  `agent.rate_limited` event (`parked_absent: "below the 90% park threshold…"`) and dispatch
+  continues. Any other status — a real wall, or one this repo has no reading for — still parks
+  unconditionally, as #298 always has. Second half: `run auto --wait-gates` (`runAuto.ts`) no
+  longer waits on a rate-limit-parked gate as if a person had to decide it — it reads the
+  park's own `resets_at` off the ledger (`src/core/run/rateLimitPark.ts`) and resumes the
+  stage itself once the provider's own clock has passed it (signed `run auto (rate-limit)`,
+  a separate actor from #231's checks-retry so the two bounds can never collide), backing off
+  a fixed interval when the frame stated no reset at all — but ONLY when that park explains
+  EVERY story the stage still has unfinished (every one a plain, never-attempted `todo`,
+  scoped to the CURRENT attempt's own `stage.started`): a gate also held by an independently
+  `blocked` story, or one whose only park is stale history from an earlier attempt, is never
+  auto-resumed (caught on review — `storiesCondition` collapses every unfinished story into
+  one `stories` id, so a park on one story is not evidence about another). `run status`'s
+  "held by: stories" line now says "parked by a rate-limit warning" and when it resumes,
+  under the same predicate, instead of reading as a decision waiting on a human.
+- **A declared `tool_restore:` command never ran anywhere automatically, so the base pre-flight's
+  own first probed command could refuse over a missing local tool before an operator had any
+  evidence beyond that refusal (#363).** Measured on a live unattended run (0.31.1, .NET
+  workspace): the workspace declared `tool_restore: dotnet tool restore` and no `install:` at
+  all — only `install:` was ever auto-run, and only inside a story's own fresh worktree
+  (`worktreeDeps.ts`) or the Build-entry probe's throwaway worktree (`entryProbe.ts`, #254),
+  never in the checkout the base pre-flight itself reads — so `dotnet-ef` stayed missing there
+  until an operator ran `dotnet tool restore` by hand. `redBaseRefusal`
+  (`src/core/build/dodRunner.ts`) now runs a repo's declared `tool_restore:` once in the checkout,
+  before its first probed command, and refuses Build by name — nothing dispatched, nothing
+  charged — when that restore itself fails. `install:` is deliberately NOT added to this same
+  door: it already has two (a story's worktree, the entry probe's worktree), and running it a
+  third time, automatically, in the human's own checkout regressed an existing one during this
+  fix (`test/story-worktree-deps.test.ts` case (d) started reporting the wrong sentence) — see
+  `toolRestoreCommandFor` in `src/core/build/worktreeDeps.ts` for why that door stays closed.
+- **The base pre-flight's own checkout and a story's worktree are different trees, and a green
+  reading from the first said nothing about it (#363).** Measured on the same live run: the base
+  pre-flight ran a command clean in the checkout; the identical command, on the identical base
+  sha, then failed inside a story's own worktree — a contract test asserting a build-time commit
+  id that SourceLink could not resolve there, an environment difference (a `.git` FILE, git's own
+  worktree convention, vs. the checkout's `.git` directory) the checkout reading cannot see by
+  construction. Moving the probe itself into a throwaway worktree was considered and rejected
+  with evidence: `dodRunner.ts`'s own "Where it runs" already documents why the checkout is
+  deliberate (a pristine worktree "would fail half the world's repos for want of `node_modules`
+  — turning this safety net into an outage"), and paying for every declared command a second time
+  on every run that needs a fresh measurement is a real, ongoing cost for a narrower class of gap
+  than that outage. Instead, absent-with-reason (AGENTS.md §7): every base-tree row in
+  `04-build/preflight.yml` now carries `tree: checkout`, so a reader — human or the loop itself —
+  cannot mistake "green on the base" for "green in the tree shape a story's own DoD actually runs
+  in".
+- **The Build developer's own brief told it "these files are the ONLY ones you may read" —
+  literally, for every story whose declared `touches` were fully inlined — and a headless
+  developer that believed it asked a question instead of opening a file it needed and left no
+  diff (#364).** Measured live: story S5's `touches` had 2 files, both inlined; `## Inputs`'
+  preamble (`src/core/facilitator/prompt.ts`) and `## Investigate` step 1 (`src/core/build/
+  prompts.ts`, "They are the whole brief") both read as a hard READ ceiling rather than the
+  WRITE allowlist `touches` is meant to be, so the developer could not see how to reach the
+  port interface, command fields and sibling-test conventions its acceptance criteria needed —
+  $0.50, no diff, and the story sat `blocked` until an operator ran `story reopen` +
+  `reject --and-continue`. `preamble()` is now role-aware (`PreambleRole`, default `"reader"`
+  byte-identical to before): the Build developer's `## Inputs` and `## Investigate` now say
+  plainly that `touches` limits what it may CHANGE, never what it may READ — every other
+  caller (Watch, the `What` stage) is unchanged. Second half of the same defect: when the
+  run's `questions_policy` (§2.2) resolves the Build stage to anything but `human`, the
+  developer brief now carries an explicit rule — "This run is unattended: nobody answers
+  questions. Decide, state the assumption in your handoff, and proceed" — and a turn that
+  asks anyway (`questions_asked` non-empty) and leaves no diff is recorded `failure_kind:
+  "asked_no_diff"` (reusing #348's vocabulary) and requeued while attempts remain, the same
+  shape a red DoD gets, instead of a terminal `blocked` only a human could lift. Attended runs
+  are unchanged: the new checks are gated on `unattended` and never fire under the default
+  `human` policy. This changes the golden developer prompt's bytes on purpose —
+  `test/build-golden.test.ts`'s fixtures are updated for exactly the `## Inputs` preamble and
+  `## Investigate` steps 1–2.
+- **A story rescued from a dead-developer block had the same gap #361 fixed for a stale
+  dependency release, one branch above it in the wave loop (#366).** Deciding
+  `blockedByFailedDeveloper(planned) !== null` — a `blocked` row whose developer never actually
+  ran — only added the story to this pass's `pending` list and logged a line; the story file
+  itself stayed `blocked` until `driveStory` settled an attempt, and `driveStory`'s very first
+  act is the same gh #298 rate-limit park #361 fixed for the dependency case, which returns
+  before any write when the wall is already up from an earlier story's turn in the same pass. A
+  story rescued and then parked before its own turn was left exactly where it started, `blocked`
+  on disk, forever. Both releases now go through one shared step, `releaseBlockedStory`
+  (`src/core/facilitator/executors/build.ts`), that writes `blocked` → `todo` the instant either
+  kind of release is decided, before the story is even offered a turn — one implementation, so a
+  park right after either can never undo it.
+
+### Changed
+
+- **`docs/ROADMAP.md` states the project's north star and how we measure toward it (owner
+  decision, 2026-09-16).** One sentence, three metrics tracked on every run (human
+  interventions per run, predicted-vs-actual cost and time, defects the reviewer catches
+  before merge versus after), a measured "where we are" for 2026-09-16, and how the shipped
+  pieces and the still-unbuilt ones map to it.
+
+### Added
+
+- **The `plan` gate refuses a story that enforces an invariant over existing data ahead of the
+  story that populates it (#365).** Measured on a live unattended run: S1 (wave 1) added a
+  database check constraint reading, in effect, "when fulfillment mode = Delivery, three columns
+  are NOT NULL" — and S4 (wave 2), the story that starts populating those columns, ran a wave
+  later. The `depends_on` graph was VALID (S4 legitimately depended on S1), so `validateWaveOrder`
+  saw nothing wrong; the defect was semantic, not structural — every existing Delivery fixture
+  violated S1's own constraint the moment S1 added it, so S1's DoD was structurally red before a
+  line of implementation. Two changes, `src/core/plan/planShape.ts`: (a) a new `PLAN_SHAPE_RULES`
+  entry tells the Plan agent a story adding an invariant over existing data lands in the same
+  story as, or in a wave after, the story that satisfies it — write it non-enforcing
+  (nullable/consistency-only) if it must land first; (b) `validatePlanShape` now scans each
+  story's `acceptance`/`test_plan` sentences for a short, exported enforcement-keyword set
+  (`ENFORCEMENT_KEYWORDS`: check constraint, not null, required, must be present, validation
+  rule, rejects, refuses) naming a field, and refuses the plan when a story no later in wave
+  order names the same field — matched case- and separator-insensitively, so `delivery_address_text`
+  and `DeliveryAddressText` collide — with a populate verb (`POPULATE_VERBS`: sets, populates,
+  writes, stores, fills, assigns) — naming both story ids, each story's own spelling, and both
+  sentences. This is the
+  cheaper substitute for re-enabling the `how` stage the owner already declined (owner decision):
+  a targeted static check over the plan's own text, not a whole extra paid agent turn. No
+  override field exists for a false positive on purpose — rewording the sentence past the
+  keyword set, or moving the story, is cheaper than a new plan-file escape hatch, and none of
+  the sibling shape rules (the wave cap's `wave_cap_reason` aside, which relaxes a different,
+  purely numeric rule) has one either.
+
 ## 0.31.1 — 2026-09-16
 
 ### Fixed

package/README.md

@@ -335,6 +335,7 @@ back on the registry is 0.3.0.
 
 | Version | Date | Status | Contains |
 |---|---|---|---|
+| 0.32.0 | 2026-09-16 | `beta` | Seven framework defects measured on the second unattended run (run #42), where every one of nine operator stalls was the framework's, not the code's. A refused compound shell line is a rework note for the developer instead of a human stop (#360); a dependency hold is re-evaluated the moment the dependency lands (#361) and a rescued story is written back to `todo` the instant the release is decided (#366); an event payload over the size cap is clamped to a bounded head instead of failing the whole build stage (#359); a headless developer is told it is unattended, that `## Inputs` is where to start and not an allowlist, and a turn that asks instead of deciding is a named `asked_no_diff` failure (#364); the rate-limit park waits for 90% of the window, not the provider's first warning at half of it (#367); the plan refuses a story that enforces an invariant on a field before the story that populates it, matching the field across spellings (#365); and the preflight restores workspace tools before its first probe and names the tree it ran in (#363, in part). Minor release: a new plan refusal, a changed park threshold and a new failure kind. |
 | 0.31.1 | 2026-09-16 | `beta` | One fix, measured on the first unattended run under 0.31.0: a seed placed at the documented `.tldrx/seeds/` location was invisible to the seed-solution detector, because the filter that drops the `what` stage's own framework inputs dropped every `.tldrx/` path, so `how` ran at full price for exactly the seeds #346 was written to skip (#358). |
 | 0.31.0 | 2026-09-16 | `beta` | Ten efficiency adjustments, ranked from a measured audit of 48 runs, so an unattended run spends less and stalls less. `how` is skipped when the seed already declares its own solution, and runs on `sonnet` when it does run (#346); `risks.md` is no longer generated (#350); `watch` follows `build`'s gate policy, so an all-auto run signs itself (#349); a Watch card's `[src:]` punctuation slip is repaired locally instead of costing a turn (#345); a fix list with zero open findings settles the round, an unreadable one holds it by name (#218); `run status` warns NO-RETRY before a retry the budget refuses (#232); a failed headless turn carries a named `failure_kind`, so the loop stops buying the same retry for every death (#348); an orphaned turn is banked as unmetered instead of a confident zero (#337); a relaunch resumes its own epic branch instead of restarting it (#347); the base-tree DoD refusal names the workspace `install:` (#343) and `run auto` names `--wait-gates` (#342). |
 | 0.30.0 | 2026-09-15 | `beta` | Seventeen changes that keep an unattended run moving instead of stranding. A cached Build refusal says so and a relaunch re-measures it (#339); a rate-limit warning is read instead of dropped, so a Build parks before the wall (#298); the plan-over-stage advisory speaks only past 2× (#302); a stage death stops recommending a retry the budget refuses (#232); a DoD command is measured twice, a fix-list finding is checked against the epic tip, and an over-long sha is refused by name (#163); a formatting slip gets one bounded repair turn (#288); five Checks stop asking the reviewer for what it can't do (#195); a Codex review no longer dies before it runs (#148); `tldrx init` probes four commands at once (#180); `tldrx ship` refuses a merged diff nobody judged (#311). Docs: the zero-touch recipe pairs `run auto` with `--wait-gates`/`--wait-answers` (#342). |

package/dist/hooks/budget-gate.js

@@ -27,7 +27,7 @@ import {
   validateRunBudget,
   wouldExceed,
   wouldExceedHostTokens
-} from "./chunk-me0kzc6k.js";
+} from "./chunk-5at77jx8.js";
 import {
   EventLog
 } from "./chunk-akemhezh.js";

package/dist/hooks/{chunk-me0kzc6k.js → chunk-5at77jx8.js}

@@ -12,6 +12,7 @@ import {
   asDocument,
   classifySrc,
   isRecord,
+  parseFrontMatter,
   parseYaml,
   readableSource,
   requireArray,
@@ -939,6 +940,15 @@ function statusOf(runDir, id) {
     return "todo";
   }
 }
+function storyDependsOn(runDir, id) {
+  try {
+    const doc = parseFrontMatter(readFileSync5(join5(runDir, PLAN_DIR, "stories", `${id}.md`), "utf8")).doc;
+    const list = doc?.depends_on;
+    return Array.isArray(list) ? list.filter((x) => typeof x === "string") : [];
+  } catch {
+    return [];
+  }
+}
 function costByStory(runDir) {
   const costs = new Map;
   const path = join5(runDir, "events.jsonl");
@@ -1441,4 +1451,4 @@ function round6(n) {
   return Math.round(n * 100) / 100;
 }
 
-export { currentActor, nowRfc3339, spentBasis, tallyOf, DEFAULT_ON_HOST_TOKENS_EXCEED, DEFAULT_ON_GRANT_EXCEED, DEFAULT_ECONOMY, economyFor, isHostTokens, validateRunBudget, asRunBudget, expertsDir, loadExperts, readExpertDomain, pathsIntersect, stackExpertNames, WAVES_FILE, BUILD_PHASE2 as BUILD_PHASE, shortBy, remainingWork, wouldExceed, wouldExceedHostTokens, raiseGrantVerdict, REBALANCE_SOURCE, planRebalance, applyRebalance, describeRebalance, raiseCommand };
+export { currentActor, nowRfc3339, spentBasis, tallyOf, DEFAULT_ON_HOST_TOKENS_EXCEED, DEFAULT_ON_GRANT_EXCEED, DEFAULT_ECONOMY, economyFor, isHostTokens, validateRunBudget, asRunBudget, expertsDir, loadExperts, readExpertDomain, pathsIntersect, stackExpertNames, WAVES_FILE, BUILD_PHASE2 as BUILD_PHASE, buildProgress, storyDependsOn, shortBy, remainingWork, wouldExceed, wouldExceedHostTokens, raiseGrantVerdict, REBALANCE_SOURCE, planRebalance, applyRebalance, describeRebalance, raiseCommand };