@pmelab/gtd 8.4.0 → 8.6.0

package/README.md

@@ -6,10 +6,10 @@
 **Chat is a terrible source of truth. Git isn't.**
 
 **gtd** is a git-aware CLI that derives the entire agentic workflow — capture,
-plan, build, test, review — from your repository state, and prints the next
-prompt for whatever agent you point at it. Every step is a commit. Tests are run
-by the tool and branched on by exit code, so the agent never grades its own
-homework.
+plan, build, check, review — from your repository state, and prints the next
+prompt for whatever agent you point at it. Every turn is a commit. The test
+command is printed for your driver to run, and the branch is decided by what the
+run leaves in the tree — so the agent never grades its own homework.
 
 No chat scrollback. No lost sessions. No infinite fix loops. Just git.
 
@@ -120,31 +120,31 @@ for). The two steering-file entries are chosen by which file you create:
 Both flows converge on the same tail: an agent hands you a `.gtd/REVIEW.md`
 checkbox review of the diff — the prompt never inlines the diff itself; it names
 the commit the changes are based at and the agent runs `git diff` to read the
-range before writing the review. While the cycle rests at that gate, gtd opens a
-**review checkout window**: HEAD is rewound to the review base with the working
-tree untouched, so the whole reviewable change shows up as ordinary uncommitted
-changes in your editor's normal git integration (and files added during the
-cycle show up as ordinary untracked files, so discarding one deletes it). The
-window closes on the next gtd invocation. Tick a box as you review each hunk
-(ticking just records "I read this"), and leave a **comment** to request
-changes: a note on a line, an inline `// TODO`-style comment in the code, or a
-direct code edit. Any comment sends a build + re-review round — an agent first
-turns your comments into an explicit instruction list, then a build turn
-implements it (a re-review then covers only the follow-through, and a hand-edit
-is treated as your own fix the agent completes without reverting your lines; a
-comment can't be silently dropped — a build turn that addresses nothing is
-refused). For the simple flow, the build turn that follows through on feedback
-and the turn that drafts the final squash message both resume the same session
-that built the feature in the first place, since the review tail is nested
-inside that build identity rather than sitting beside it. Ticking every box with
-no comment is the sign-off, which collapses the whole cycle into one commit (a
-**squash finale** whose message an agent drafts). Stepping with a box still
-unticked and no comment is refused (finish reviewing first), as is deleting
-`.gtd/REVIEW.md`.
+range before writing the review. While the process rests at that gate, gtd opens
+a **review checkout window**: HEAD is rewound to the review base with the
+working tree untouched, so the whole reviewable change shows up as ordinary
+uncommitted changes in your editor's normal git integration (and files added
+during the process show up as ordinary untracked files, so discarding one
+deletes it). The window closes on the next gtd invocation. Tick a box as you
+review each hunk (ticking just records "I read this"), and leave a **comment**
+to request changes: a note on a line, an inline `// TODO`-style comment in the
+code, or a direct code edit. Any comment sends a build + re-review round — an
+agent first turns your comments into an explicit instruction list, then a build
+turn implements it (a re-review then covers only the follow-through, and a
+hand-edit is treated as your own fix the agent completes without reverting your
+lines; a comment can't be silently dropped — a build turn that addresses nothing
+is refused). For the simple flow, the build turn that follows through on
+feedback and the turn that drafts the final squash message both resume the same
+session that built the feature in the first place, since the review tail is
+nested inside that build identity rather than sitting beside it. Ticking every
+box with no comment is the sign-off, which collapses the whole process into one
+commit (a **squash finale** whose message an agent drafts). Stepping with a box
+still unticked and no comment is refused (finish reviewing first), as is
+deleting `.gtd/REVIEW.md`.
 
 The same review tail also has a direct entry point —
 `gtd --entry review-gate.check --var reviewBase=<commitish>` starts a brand new
-process reviewing `<commitish>..HEAD` with no cycle of its own, e.g. a
+process reviewing `<commitish>..HEAD` with no build of its own, e.g. a
 colleague's PR branch (`review-gate.check`'s `reviewBase:` is a template bound
 to the `reviewBase` var, so supplying it via `--var` fixes the whole process's
 diff base to that commitish). Its squash keeps and describes only the fixes made
@@ -207,14 +207,12 @@ Commands:
                    just-finished invocation's token cost and model on the
                    turn commit (summed into it.processCost/processCostByModel).
                    Pass --entry <state> to start a brand NEW process at
-                   <state> instead — any declared, non-commit state (e.g.
-                   review-gate.check or fix-precheck on the bundled unified
-                   template) — with repeatable --var <name>=<value> supplying
-                   that new process's fixed it.vars overrides
+                   <state> instead — any declared, non-commit state — with
+                   repeatable --var <name>=<value> supplying that new
+                   process's fixed it.vars overrides
   (no command) --entry <state>
                    Short form of 'step human --entry <state>' — starts a new
-                   process authenticated as human, e.g.
-                   'gtd --entry review-gate.check'
+                   process authenticated as human, e.g. 'gtd --entry <state>'
   abandon          End the process currently underway without completing it:
                    close any open review checkout window, then rewind HEAD to
                    the commit the process started from, keeping everything it
@@ -226,17 +224,29 @@ Commands:
                    Refuses on a dirty working tree, when there is no retained
                    history, or when HEAD has advanced past the squash with
                    commits that would be lost
-  next             Print the resolved rest's rendered script/prompt/message
-                   (no mutation)
+  next             Print the resolved rest's rendered script/prompt/message (no mutation)
   status           Print the resolved rest's state/actor and which declared
                    pattern (if any) each pending change matches (no mutation)
-  validate         Format and validate the steering file the resolved rest
-                   declares, with its mode's commands (its file:/mode:);
-                   exits non-zero with the findings when it is invalid
+  validate         Print the script that formats (when declared) then
+                   validates the resolved rest's steering file, using its
+                   mode's commands (its file:/mode:), instead of running it —
+                   a driver runs the script and reads the findings from its
+                   own exit code/output. Always exits 0; --json emits
+                   {state, file?, mode?, script} (script is "" when there is
+                   nothing to validate; plain text prints "nothing to
+                   validate" in that case)
   lsp              Start the LSP server for .gtd/ steering files (stdio)
   visualize        Serve an interactive diagram of the active workflow on a
                    local web server (--port <n>, --no-open; --json prints the
                    model and exits)
+  check <mode> <file>
+                   Read <file> and run the built-in steering format named
+                   <mode> (see `gtd validate`'s modes: qa, review) over its
+                   contents, printing each finding one per line and exiting
+                   non-zero when there are any. Resolves no workflow state and
+                   reads no config — standalone, runnable from any directory
+                   with <mode>/<file> given explicitly. This is what a
+                   workflow's emitted validation script invokes as a leaf step
   version          Print version and exit
   help             Print this help and exit
 
@@ -255,9 +265,6 @@ Options:
                    (with --entry; repeatable) supply a fixed it.vars
                    override for the new process; the name must already be
                    declared by the workflow's own vars: or the .gtdrc vars:
-  --once           (bare gtd or gtd loop only) run exactly one loop beat (one
-                   human-gate capture, one script check+step, or one agent
-                   prompt+step), then exit
   --version, -v    Print version and exit
   --help, -h       Print this help and exit
 ```
@@ -312,9 +319,16 @@ envelope on **stdout**, and still exits 1:
 { "state": "error", "prompt": "<message>" }
 ```
 
+This covers every failure mode, not just a command's own refusal: a **usage
+error** (an unknown flag, a missing argument, `gtd --entry version`'s "not an
+enterable state") and a **defect** (a layer throwing outside the ordinary error
+channel) both get the same envelope — there is no failure path that reaches
+`--json` without one.
+
 A human-readable `gtd: <message>` line is still written to **stderr** regardless
 of `--json` — the envelope adds a structured stdout channel, it does not replace
-the plain-text one.
+the plain-text one. Stderr always carries exactly one `gtd: ` prefix: a message
+already authored with its own `gtd:`/`gtd <cmd>:` prefix is never doubled.
 
 ## Driving the loop
 
@@ -453,7 +467,7 @@ A few things to know before relying on it:
   owns a `herdr:claude` session until the process exits; while it does, the
   wrapper's reports are silently ignored and the sidebar stops tracking `gtd`.
 - **`blocked` covers the resting `idle` state too** — gtd's own `idle` is a
-  human gate (it waits for you to write a steering file), so a finished cycle
+  human gate (it waits for you to write a steering file), so a finished process
   reads as "your turn", which is what it is.
 - The status **persists after the wrapper exits** — that's the point: the
   sidebar keeps showing which worktree is waiting on you. Hand the pane back to
@@ -468,6 +482,123 @@ Optionally,
 sets a value renderable as `$summary` in an Agent sidebar row, if you want more
 than the state itself.
 
+## Writing your own driver
+
+`bin/gtd` is gtd's own reference driver, not a privileged one — the engine
+itself is a supported public surface, and anything below holds for any driver
+you write against it. gtd decides and prints; it never touches git itself. The
+four commands that change anything — `gtd step [<actor>]` (including
+`--entry <state>`), `gtd abandon`, and `gtd restore` — perform no git write when
+run. Instead, under `--json`, each one's output object carries two extra string
+fields, `required` and `optional`: bash scripts for YOU to execute. (Plain-text
+output, without `--json`, never shows these — it prints only the human-facing
+result line, e.g. `committed: <subject>`, so a plain-text caller never sees a
+script at all.) A driver that only prints gtd's plain-text output is not driving
+anything; to actually land a turn, run `gtd ... --json`, pull `required`/
+`optional` out with `jq`, and execute them.
+
+- **`required`** is everything that decides what lands in git — closing an open
+  review checkout window, the resting state's own steering-mode
+  `format:`/`validate:` commands, then the commit or squash itself (`gtd step`
+  and `gtd --entry <state>`), or the ref update and reset that undo a process
+  (`gtd abandon`, `gtd restore`). Run this one. Skipping it means the turn never
+  lands.
+- **`optional`** is presentation only: re-opening the review checkout window
+  (the `<<<<<<< HEAD` diff view) after a `gtd step` lands at a
+  `reviewWindow: true` state, so an editor's diff view has something to show.
+  Skip it and you lose nothing but that view — the workflow is still driven
+  correctly. `gtd abandon`/`gtd restore` always emit `optional: ""` (there is no
+  window to reopen after either).
+
+Either field may be the empty string `""`, meaning "nothing to do here" — a
+no-op `gtd step` (a clean tree matching no `on` pattern) emits
+`required: ""`/`optional: ""` and exits zero; run nothing and move on. Both
+scripts are self-contained (each carries its own precondition assert and retry
+helper) and safe to run standalone, in sequence, or not at all — paste either
+into a terminal and it does exactly what it says.
+
+### Failure taxonomy and recovery
+
+Two different things can go wrong, and they mean different things:
+
+- **`gtd` itself exits non-zero.** Nothing was attempted — this is a refusal (a
+  guard rejected the turn) or a usage error. No script was ever produced.
+- **An emitted script exits non-zero when YOU run it.** Something may have
+  partially happened — e.g. a `gtd_retry`-wrapped git write landed but a later
+  step in the same script failed.
+
+Recovery is the same in both cases: **re-invoke `gtd`.** It re-reads the real
+repository state fresh every time — never a cached plan — and emits whatever
+still needs to happen from there. This works because every emitted script opens
+by asserting its own precondition
+(`[ "$(git rev-parse --verify --quiet HEAD 2>/dev/null)" = <expected> ] || { ...; exit 1; }`,
+and the same shape for a review window's saved ref — see `src/Emit.ts`'s
+`headAssertion`/`reviewWindowAssertion`), so a script generated against a
+repository state that has since moved refuses loudly instead of corrupting
+anything. In a repository with no commits yet, `<expected>` is the empty string
+— the deciding read's own convention for "no commits yet" — so the first
+workflow commit is allowed to land instead of being blocked by an unborn `HEAD`.
+**Emitted scripts are re-runnable**: this is the single most important property
+for a driver's recovery logic. Re-running a script that already fully applied is
+a no-op (its git writes are `--allow-empty` commits, idempotent ref updates, and
+tolerant staged-restore calls), and re-running one that only partially applied
+resumes correctly, because the precondition either still holds (nothing landed
+yet — safe to retry verbatim) or gtd's next invocation reads the new real state
+and emits a fresh script for what remains. A driver never needs its own
+retry/resume logic beyond "if the script failed, ask gtd again."
+
+### Prerequisites
+
+- **`jq`** — to pull the `required`/`optional` strings back out of gtd's
+  `--json` output.
+- **`gtd` on `PATH`** — a mode's seeded `validate:` command (the one the
+  compiler fills in for the built-in `qa`/`review` formats) is literally the
+  string `gtd check <mode> '<file>'`, invoked by NAME from inside an emitted
+  script, not by absolute path (see `src/SteeringFormats.ts`'s
+  `seededValidateCommand`). This is a deliberate trade: a readable, overridable,
+  copy-pasteable command in exchange for depending on shell name resolution at
+  the moment the script runs. The sharp edge: if the `gtd` binary you invoked to
+  GENERATE the script differs from the `gtd` that resolves on `PATH` when the
+  script later RUNS (a locally-built dev binary vs. a globally-installed
+  release, say), you can get version skew between the two — the command that
+  validates may not be the command that decided. Keep the two in sync (one `gtd`
+  on `PATH`, consistently) if you care about that gap.
+
+### The normalization-only contract on `format:`
+
+A mode's `format:` command may reformat a steering file — whitespace, wrapping,
+reordering — but must NEVER change what a step-capture guard would decide. gtd's
+guards (the review sign-off check, the feedback-progress check, the
+answer-completeness check — `src/StepGuards.ts`) decide ONCE, against whichever
+bytes are on disk at the moment `gtd step` runs, which may be before OR after an
+emitted script's own `format:` line has run (the script runs `format:` then
+`validate:` then the commit — see `src/SteeringMode.ts`'s
+`renderSteeringCommands` — but gtd's decision and the driver's script execution
+are different processes at different times, so there's no guaranteed ordering
+between "gtd decided" and "the script formatted"). That's only safe because
+every built-in guard judges content, not incidental formatting (e.g. it
+normalizes `[ ]`/`[x]` checkboxes before comparing). If you plug in your own
+`format:` command, the same rule binds it: a formatter that also changes meaning
+— ticking a box, stripping a paragraph — makes the guard's decision and the
+file's actual content disagree, and gtd will not catch that for you.
+
+### Built-in steering formats are ordinary modes
+
+`qa` and `review` are gtd's two built-in steering-file formats (parsed and
+validated in-process because `gtd lsp` needs the same parsers for live
+diagnostics), but their `validate:` is not hardcoded or hidden: the compiler
+SEEDS every workflow's `modes:` map with `qa`/`review` entries whose `validate:`
+is the same `gtd check <mode> '<file>'` string described above
+(`src/PatternConfig.ts`, `src/SteeringFormats.ts`'s `seededValidateCommand`).
+That seeded command is visible in `gtd visualize`'s compiled model and in the
+editor JSON schema like any other mode, and it's overridable the same way any
+mode is — declare `modes: { qa: { validate: "your-own-command" } }` (in the
+workflow or in `.gtdrc`) and your command displaces the seed; declaring only a
+`format:` for `qa`/`review` composes with the seeded `validate:` rather than
+replacing it. There is no special-cased built-in behavior a driver needs to know
+about beyond the ordinary mode-resolution rules already documented under
+[The `workflow:` key](#the-workflow-key).
+
 ## Configuration
 
 gtd reads an optional `.gtdrc` config file via
@@ -527,7 +658,7 @@ workflow:
     anyKey: anyScalarValue
   modes: # optional — steering-file modes a state's `mode:` may name
     <name>:
-      format: <shell command> # at least one of format/validate
+      format: <shell command> # both optional — {} is the format-only tier
       validate: <shell command>
   entry:
     default: <machine name> # which machine is the ROOT instance
@@ -553,11 +684,14 @@ workflow:
           retry:
             max: <number>
             otherwise: <targetState>
+          label: <string> # optional, opaque display name passed through `gtd next --json`/`gtd status --json`
           file: <string> # optional, an Eta template naming the state's steering file
-          mode: <modeName> # optional, requires "file" — a built-in (qa/review/prose) or a `modes:` entry
+          mode: <modeName> # optional, requires "file" — must be declared in `modes:` (qa/review are seeded for you; everything else, including prose, you declare)
           reviewWindow: true # optional — open the review checkout window at rest here
           reviewBase: true # optional — anchor the review window's diff base to this state's most-recent commit
           # reviewBase: <Eta template> # OR a template — rendered (only meaningful entering via --entry) to a commitish that fixes the WHOLE PROCESS's diff base
+          requireProgress: true # optional, requires "file" — refuse a turn whose only change deletes this state's own `file:`
+          answerGate: true # optional, requires "file" — refuse a turn until every open question in the (qa-mode) `file:` is answered
           entry: true # optional — an EXTRA reachability root (`entries.manual`), enterable via `gtd --entry <this state's qualified name>` — NOT a precondition for `--entry` (any declared, non-commit state is a valid target)
         <local>: { machine: <name>, with: { <param>: <value> } } # a REFERENCE — instantiates <name> as a child, qualified as `<local>.<childLocal>`
 ```
@@ -587,7 +721,7 @@ conversation across both call sites.
 
 Besides `it.vars` (below), a `script`/`prompt`/`message`/`commit` template sees:
 
-- **`it.startCommit`** — the process's diff base (the commit the current cycle
+- **`it.startCommit`** — the process's diff base (the commit the current process
   started from, or the base a `--var reviewBase=<commitish>` entry resolved to).
 - **`it.reviewBase`** — the previous review round's boundary, falling back to
   `it.startCommit` on a first review.
@@ -607,9 +741,9 @@ model, pattern grammar, load-time rules, and how to verify a change compiles.
 
 > **Upgrading from a pre-8.2 `workflow:`?** The old flat `states:` shape (with a
 > per-state `initial: true`/`reviewEntry: true`/`fixEntry: true` flag) is no
-> longer accepted — finish or `gtd abandon` any in-flight cycle before
-> upgrading, since the old and new shapes aren't compatible mid-cycle. Wrap your
-> states under a single
+> longer accepted — finish or `gtd abandon` any in-flight process before
+> upgrading, since the old and new shapes aren't compatible mid-process. Wrap
+> your states under a single
 > `machines: { <name>: { entry: <initial state>, states: {...} } }` and declare
 > `entry: { default: <name> }` at the top level (moving any
 > `reviewEntry`/`fixEntry` state to a plain per-state `entry: true` flag,
@@ -785,13 +919,18 @@ own `- [ ]` line — diagnostics for both (live as you edit), and a
 
 It is config-driven via each state's `file:`/`mode:`, and falls back to basename
 dispatch (`REVIEW.md` → `review`) with no config in sight. `qa` and `review` are
-gtd's built-in steering-file MODES with a VALIDATOR gtd itself implements; a
-mode's `format:` and `validate:` are shell commands a workflow (or a project's
-`.gtdrc`) declares for itself, so you bring your own formatter and your own
-checkers. A third built-in, `prose` — format-only, no validator, used by the
-simple flow's plan file — has no live editor support: `gtd lsp` publishes no
-symbols or diagnostics for it, though `gtd validate` and the `gtd step` gate
-still format and validate it like any other mode.
+gtd's built-in steering-file FORMATS, each with its own outline/actions/
+go-to-definition and a VALIDATOR gtd itself implements; a mode's `format:` and
+`validate:` are shell commands a workflow (or a project's `.gtdrc`) declares for
+itself, so you bring your own formatter and your own checkers. Overriding one of
+`qa`/`review`'s `validate:` with a shell command does NOT lose its outline/
+actions — those come from the FORMAT (the name), independent of who validates —
+but `gtd lsp` never runs a shell command per keystroke, so live diagnostics
+become one `Information` notice pointing at `gtd validate` instead of the
+built-in findings. Any OTHER mode name — `prose` (used by the simple flow's plan
+file), or a project's own declared name — has no built-in format, so it gets no
+live editor support at all: `gtd validate`'s emitted script and the `gtd step`
+gate still format and validate it like any other mode.
 
 ## Development
 
@@ -800,8 +939,10 @@ npm install
 npm run dev          # run from source, no build (node dev/run.mjs)
 npm run build        # tsdown → dist/gtd.bundle.mjs
 npm test             # format:check, typecheck, lint, unit + e2e tests, fallow
-npm run test:unit    # vitest unit tests (the pure resolver) — --project unit
-npm run test:e2e     # gherkin e2e via vitest + quickpickle — --project e2e
+npm run test:unit       # vitest unit tests (the pure resolver) — --project unit
+npm run test:e2e        # gherkin e2e, both tiers — test:e2e:inmem + test:e2e:live
+npm run test:e2e:inmem  # @inmem scenarios, in-process, no build — --project e2e-inmem
+npm run test:e2e:live   # @live scenarios against the built bundle — --project e2e-live
 npm run test:mutation # StrykerJS mutation testing (manual only, ~10 min)
 npm run typecheck
 npm run lint
@@ -817,7 +958,20 @@ The decision core is pure and IO-free: the pattern machine's shape (states,
 patterns, retry) is a plain-data `WorkflowDefinition` (`src/PatternMachine.ts`),
 and the same module's `resolveState`/`step` are the pure resolver/interpreter
 over it — so the whole engine is trivially unit-testable in isolation. All
-git/filesystem/template IO is confined to the edge (`src/Edge.ts`).
+git/filesystem/template IO is confined to the edge (`src/Edge.ts`), whose
+`currentRest` resolves everything derivable from where the process rests right
+now into a single snapshot, and whose `planStep`/`planEntry` pair separates
+deciding a transition from performing it — so a command's own capture guards run
+between the two. The edge reaches the outside world through two content ports
+(`src/RepoFiles.ts` for working-tree and committed reads, `src/CommandRunner.ts`
+for the one place gtd spawns a subprocess itself), with the `gtd step` capture
+guards (`src/StepGuards.ts`) built on top of them.
+
+`src/testing/` holds the in-memory git/config/filesystem test double every
+`@inmem` scenario and unit test runs against instead of a real repo — it never
+ships (a lint rule and a build-time check both enforce that) and is trustworthy
+only because the same `GitOperations` contract suite runs against a real git
+repo too.
 
 Releases are automatic: push releasable Conventional Commits (`fix:`, `feat:`,
 or breaking changes) to `main` and semantic-release computes the next version,