@pmelab/gtd 3.4.0 → 3.6.0

package/docs/cli.md

@@ -6,13 +6,20 @@ Usage: gtd [command] [options]
 Commands:
   step <actor>     Authenticate as <actor>, match the resolved rest's
                    declared patterns against the pending changes, and commit
-                   (or squash) the one resulting transition
+                   (or squash) the one resulting transition. Pass
+                   --cost=<n> (optionally --model=<name>) to record the
+                   just-finished invocation's token cost and model on the
+                   turn commit (summed into it.processCost/processCostByModel)
   next             Print the resolved rest's rendered script/prompt/message
                    (no mutation)
   run              Execute the resolved rest's emitted script, then step its
                    actor (the built-in script driver)
   status           Print the resolved rest's state/actor and which declared
                    pattern (if any) each pending change matches (no mutation)
+  validate         Validate the steering file the resolved rest declares
+                   (its file:/mode:) against that mode's format; exits
+                   non-zero with findings when the file violates it (no
+                   mutation)
   mermaid          Print the active workflow's shape as Mermaid
                    stateDiagram-v2 source (no mutation)
   format <file>    Format a markdown file in place
@@ -20,6 +27,8 @@ Commands:
 
 Options:
   --json           Output structured JSON instead of plain text
+  --cost=<n>       (gtd step only) record the invocation's token cost
+  --model=<name>   (gtd step only, with --cost) tag that cost's model
   --version, -v    Print version and exit
   --help, -h       Print this help and exit
 ```
@@ -32,11 +41,15 @@ without touching the repository. Every other command must be run from the
 history relative to cwd, so it refuses with a clear error if invoked from a
 subdirectory.
 
-`--json` is the only long option. Any other `--` option (including a typo like
-`--jsn`) is rejected with a usage error rather than silently ignored, so a
-mistyped flag can never degrade a JSON caller to plain-text mode.
+`--json`, `--cost=<n>`, and `--model=<name>` (the latter two only for
+`gtd step`) are the only long options. Any other `--` option (including a typo
+like `--jsn`) is rejected with a usage error rather than silently ignored, so a
+mistyped flag can never degrade a JSON caller to plain-text mode. A bare
+`--cost`/`--model` with no value, a non-numeric or negative `--cost`, an empty
+`--model`, `--model` without `--cost`, or either flag on any command other than
+`gtd step` are all usage errors.
 
-## `gtd step <actor>`
+## `gtd step <actor> [--cost=<n>] [--model=<name>]`
 
 Authenticates `<actor>` against the resolved rest and performs the ONE resulting
 transition. Unlike v2, there is no fixpoint chain to drive: the pattern
@@ -45,6 +58,16 @@ authors at most one commit (a normal turn) or performs one squash
 (§[STATES.md](../STATES.md#8-the-squash-lifecycle)) — never both, never a chain
 of several. A caller that wants several transitions issues several invocations.
 
+`--cost=<n>` records the token cost of the invocation that produced the pending
+changes (a non-negative number), and the optional `--model=<name>` tags it with
+the model that ran, as a `Gtd-Cost: <n> <model>` trailer on the turn commit —
+persisted in the git log, summed across the process into `it.processCost` and
+grouped by model into `it.processCostByModel` (see
+[Configuration: Token cost](configuration.md#token-cost)). `--model` requires
+`--cost`. On a squash they are folded into the process total/breakdown the
+`commit:` template renders rather than written as a trailer. Both are orthogonal
+to `--json`.
+
 - **Out-of-turn refusal** — `<actor>` isn't the resolved state's declared actor:
   exit non-zero, zero commits.
   `gtd step <actor>: out of turn — "<state>" awaits <awaited-actor>`
@@ -69,10 +92,17 @@ or, at a no-op:
 nothing to do at "idle"
 ```
 
-`--json` emits `{state, subject}` — `subject` is `null` at a no-op:
+`--json` emits `{state, subject}` — `subject` is `null` at a no-op — plus
+`cost`/`model` keys echoing what `--cost`/`--model` recorded (each omitted when
+not passed):
 
 ```json
-{ "state": "grilling", "subject": "gtd(human): grilling" }
+{
+  "state": "grilling",
+  "subject": "gtd(human): grilling",
+  "cost": 1450,
+  "model": "claude-opus-4-8"
+}
 ```
 
 ## `gtd next [--json]`
@@ -171,15 +201,21 @@ Pending:
 or, on a clean tree: `Pending: (clean)`. A `Model: <value>` line appears right
 after `Awaits:` when the resolved state declares a `model:` hint, and
 `Memory: <value>`/`File: <value>`/`Mode: <value>` lines appear after that (in
-that order) when declared — each independently, only when set.
+that order) when declared — each independently, only when set. A `Cost: <n>`
+line appears after those when the current process has accumulated any
+`Gtd-Cost:` (see [`gtd step --cost`](#gtd-step-actor---costn-model-name)),
+omitted when the running total is `0`; when the breakdown adds information (more
+than one model, or a single tagged model), indented `  <model>: <cost>` lines
+follow it.
 
 `--json` emits
-`{state, actor, changes: [{status, path, pattern}], model?, memory?, file?, mode?, edges?}`
+`{state, actor, changes: [{status, path, pattern}], model?, memory?, file?, mode?, cost?, costByModel?, edges?}`
 — `pattern` is `null` when no declared row matches that change;
 `model`/`memory`/`file`/`mode` are present only when the resolved state declares
-them (omitted entirely, never `null`, otherwise); `edges` is the resting state's
-`on` edges as `[{ pattern, target, describe? }]` (same as `gtd next --json`),
-omitted when the state has no `on`:
+them; `cost`/`costByModel` (the `[{model, cost}]` breakdown) only when the
+running process total is above `0`; and `edges` is the resting state's `on`
+edges as `[{ pattern, target, describe? }]` (same as `gtd next --json`), omitted
+when the state has no `on` (all omitted entirely, never `null`, otherwise):
 
 ```json
 {
@@ -194,6 +230,43 @@ omitted when the state has no `on`:
 
 `gtd status` takes no arguments — extra positional args are rejected.
 
+## `gtd validate [--json]`
+
+Format **and** validate the steering file the resolved rest declares. It
+resolves the current state exactly like `gtd status`, renders that state's
+`file:`, **formats that file in place** (the same markdown formatter as
+`gtd format`), then runs the parser its `mode:` selects over the formatted
+contents — `qa` → `src/OpenQuestions.ts`, `review` → `src/ReviewDoc.ts` (the
+same pure parsers the LSP publishes as diagnostics, so there is one source of
+truth per format).
+
+- A well-formed file exits `0` (`<file>: valid`, or `{"valid":true, ...}` with
+  `--json`) — and is left formatted.
+- Violations exit **non-zero** with the findings, one per line — the signal a
+  producing agent, or the driving loop, loops on until the file is well-formed.
+- A state that declares no `file:`/`mode:`, or whose file is **absent** (e.g.
+  `building` deleted `.gtd/TODO.md`, or a human deleted `.gtd/REVIEW.md` to
+  approve), has nothing to validate and exits `0`
+  (`nothing to validate at "<state>"`). Nothing is formatted in that case.
+
+This is how the bundled default keeps its steering files tidy and well-formed
+without an in-machine validation state — and the check runs whoever last touched
+the file, agent or human:
+
+- **The producing agent self-validates.** Plain `gtd next` appends a "run
+  `gtd validate` and fix all violations" instruction to a `prompt` rest that
+  declares `file:`+`mode:`; `gtd next --json` withholds it and the driving loop
+  runs `gtd validate` after the turn instead (see `bin/gtd-loop` /
+  [STATES.md §12](../STATES.md#12-steering-file-validation-gtd-validate)).
+- **`gtd step` enforces the same gate.** Capturing a turn out of a state that
+  declares `file:`+`mode:` formats that file in place and validates it first,
+  and **refuses the step** (committing nothing) when it is invalid — so a
+  human's edit at a gate (answering at `grilling-answer`, reviewing at
+  `await-review`) is formatted and checked exactly like an agent's draft, and a
+  malformed steering file is never committed.
+
+`gtd validate` takes no arguments.
+
 ## `gtd mermaid`
 
 Pure emitter of the active workflow's **shape** — not the resolved rest — as
@@ -228,7 +301,7 @@ Mermaid-aware renderer (GitHub, GitLab, VS Code, Obsidian, the
 
 State names are aliased to Mermaid-safe identifiers (non-word characters fold to
 `_`; a digit-led name gets an `s_` prefix) via a `state "<name>" as <alias>`
-declaration up front, so a hyphenated name like `todo-validating` still displays
+declaration up front, so a hyphenated name like `grilling-answer` still displays
 with its exact declared spelling. Rejects `--json` (exit 1,
 `gtd mermaid does not accept --json`) — there is no structured shape to emit
 beyond the Mermaid source itself — and takes no arguments (extra positional args
@@ -256,10 +329,9 @@ Errors (all exit 1, message on stderr):
 Starts an LSP server over stdio for `.gtd/` steering files — document symbols
 for a `qa`-mode file's open questions and a `review`-mode file's review
 chunks/hunks, code actions to check/uncheck a hunk or a whole chunk, and
-diagnostics publishing the same parser findings the bundled workflow's
-`.gtd/FORMAT.md` validators produce (see `src/OpenQuestions.ts` /
-`src/ReviewDoc.ts` and
-[STATES.md §10](../STATES.md#10-the-bundled-default-workflow)).
+diagnostics publishing the same parser findings `gtd validate` reports (see
+`src/OpenQuestions.ts` / `src/ReviewDoc.ts` and
+[STATES.md §12](../STATES.md#12-steering-file-validation-gtd-validate)).
 
 **Config-driven** (see
 [docs/design/state-file-association.md](design/state-file-association.md)): the

package/docs/configuration.md

@@ -322,17 +322,66 @@ head is preserved under `refs/gtd/review-head` (the base under
 Every `script`/`prompt`/`message`/`commit`/`model`/`memory`/`file` template is
 rendered as an Eta template (`it.<name>`) with:
 
-| Variable         | Meaning                                                                                                                                                                                                                                 |
-| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `startCommit`    | The hash the current process started from (before its first turn).                                                                                                                                                                      |
-| `currentCommit`  | HEAD's hash at render time.                                                                                                                                                                                                             |
-| `previousCommit` | The hash before the last transition (HEAD's parent, in-process).                                                                                                                                                                        |
-| `state`          | The state whose content is being rendered.                                                                                                                                                                                              |
-| `actor`          | The actor this render is for.                                                                                                                                                                                                           |
-| `processDiff`    | `startCommit..HEAD` plus the pending working-tree diff.                                                                                                                                                                                 |
-| `lastDiff`       | The diff of the last transition alone.                                                                                                                                                                                                  |
-| `read(path)`     | Reads a working-tree file (pending contents, not HEAD's) by repo-relative path. Throws for a missing/unreadable path — for a `commit:` template, that throw refuses the step (see [STATES.md §8](../STATES.md#8-the-squash-lifecycle)). |
-| `vars`           | The merged three-layer variable map, always a flat `Record<string, string>` — see ["Variables"](#variables) below.                                                                                                                      |
+| Variable             | Meaning                                                                                                                                                                                                                                                                                                                                    |
+| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `startCommit`        | The hash the current process started from (before its first turn).                                                                                                                                                                                                                                                                         |
+| `currentCommit`      | HEAD's hash at render time.                                                                                                                                                                                                                                                                                                                |
+| `previousCommit`     | The hash before the last transition (HEAD's parent, in-process).                                                                                                                                                                                                                                                                           |
+| `state`              | The state whose content is being rendered.                                                                                                                                                                                                                                                                                                 |
+| `actor`              | The actor this render is for.                                                                                                                                                                                                                                                                                                              |
+| `processDiff`        | `startCommit..HEAD` plus the pending working-tree diff.                                                                                                                                                                                                                                                                                    |
+| `lastDiff`           | The diff of the last transition alone.                                                                                                                                                                                                                                                                                                     |
+| `processCost`        | The accumulated token cost of the current process — the sum of every `Gtd-Cost:` trailer recorded by `gtd step <actor> --cost=<n>` across its turn commits, plus the in-flight step's own `--cost` (so a `commit:` squash template sees the whole-process total). A number, `0` when none recorded. See ["Token cost"](#token-cost).       |
+| `processCostByModel` | The same accumulated cost broken down per model: an array of `{ model, cost }`, highest-cost first, one entry per distinct `--model` tag (costs recorded with no `--model` group under `"unspecified"`). A squash template iterates it to itemize the feature's cost per model. Empty when none recorded. See ["Token cost"](#token-cost). |
+| `read(path)`         | Reads a working-tree file (pending contents, not HEAD's) by repo-relative path. Throws for a missing/unreadable path — for a `commit:` template, that throw refuses the step (see [STATES.md §8](../STATES.md#8-the-squash-lifecycle)).                                                                                                    |
+| `vars`               | The merged three-layer variable map, always a flat `Record<string, string>` — see ["Variables"](#variables) below.                                                                                                                                                                                                                         |
+
+### Token cost
+
+A loop driver that knows how many tokens the invocation it just drove cost — and
+on which model — can record both with
+`gtd step <actor> --cost=<n> [--model=<name>]` (`<n>` a non-negative number).
+gtd appends them to that turn's commit as a `Gtd-Cost: <n> <model>` trailer — a
+blank line then the trailer, below the untouched `gtd(<actor>): <state>` subject
+— so the per-turn cost and model are persisted in the git log:
+
+```
+gtd(agent): reviewing
+
+Gtd-Cost: 1450 claude-opus-4-8
+```
+
+`--model` is optional and requires `--cost` (a model tag with no token count
+records nothing to sum); a cost recorded without a model is grouped under
+`"unspecified"`. `gtd status` shows the process's running total and, when the
+breakdown adds information, a per-model split (a `Cost:` line with indented
+`<model>: <cost>` lines / a `cost` number and a `costByModel` array, omitted
+when nothing has been recorded). `gtd step --json` echoes what it recorded
+(`{ "state": …, "subject": …, "cost": 1450, "model": "claude-opus-4-8" }`).
+
+Every template sees the running total as `it.processCost` and the per-model
+breakdown as `it.processCostByModel` (the table above): the current process's
+turn-commit entries, plus the in-flight step's own `--cost`/`--model`. Their
+intended use is a `commit:` squash template, which renders against the pending
+tree as the process collapses — so the whole feature's total AND its per-model
+itemization (including the squashing step itself) land in the squash message
+even though the intermediate per-turn trailers are discarded with the turns they
+rode on:
+
+```yaml
+done:
+  commit: |
+    feat: ship it
+
+    Total token cost: <%= it.processCost %>
+    <% it.processCostByModel.forEach(function(m){ %>
+    - <%= m.model %>: <%= m.cost %>
+    <% }) %>
+```
+
+`--cost`/`--model` are accepted only by `gtd step` (a usage error on any other
+command), and gtd never interprets the number's unit (tokens, cents, whatever
+the driver records) or the model name — it only records, sums, and groups them.
 
 ## Variables
 

package/docs/design/state-file-association.md

@@ -45,14 +45,14 @@ vars:
   feedbackFile: .gtd/FEEDBACK.md
 ```
 
-Per-state mapping (12 states):
+Per-state mapping (10 states):
 
-| States                                                              | `file:`                       | `mode:`                   |
-| ------------------------------------------------------------------- | ----------------------------- | ------------------------- |
-| `grilling`, `todo-validating`, `grilling-answer`, `building`        | `<%= it.vars.todoFile %>`     | `qa`                      |
-| `reviewing`, `review-validating`, `await-review`, `review-deciding` | `<%= it.vars.reviewFile %>`   | `review`                  |
-| `fixing`, `escalate`                                                | `<%= it.vars.feedbackFile %>` | — (plain text, no format) |
-| `idle`, `checking`                                                  | —                             | —                         |
+| States                                         | `file:`                       | `mode:`                   |
+| ---------------------------------------------- | ----------------------------- | ------------------------- |
+| `grilling`, `grilling-answer`, `building`      | `<%= it.vars.todoFile %>`     | `qa`                      |
+| `reviewing`, `await-review`, `review-deciding` | `<%= it.vars.reviewFile %>`   | `review`                  |
+| `fixing`, `escalate`                           | `<%= it.vars.feedbackFile %>` | — (plain text, no format) |
+| `idle`, `checking`                             | —                             | —                         |
 
 Prompts and scripts that already name these files switch to the vars
 (`<%~ it.vars.todoFile %>` etc.) so each filename has ONE source of truth in
@@ -109,4 +109,5 @@ The advanced example (`docs/examples/advanced-workflow.md`) gains the same
 - Eta-rendering of `on` pattern keys (the rename-switch story) — future work,
   noted in §2.
 - New LSP modes beyond `qa`/`review` — the vocabulary can grow with new formats;
-  adding one means: parser (spec), validator recipe, LSP dispatch arm, docs.
+  adding one means: parser (spec), a `gtd validate` dispatch arm for the mode,
+  LSP dispatch arm, docs.

package/docs/design/state-machine-validation.md

@@ -51,7 +51,7 @@ Each of these passes today's validation:
    `retry: {max: 3, otherwise: escalate}`; delete that key and the machine
    validates clean but can ping-pong forever with no human in the loop.
 5. **Shadowed `on` row** — `on` is first-match-wins, so
-   `{"* **": a, "A .gtd/FORMAT.md": b}` never routes to `b`: the catch-all
+   `{"* **": a, "A .gtd/FEEDBACK.md": b}` never routes to `b`: the catch-all
    subsumes the specific row. Dead rows are silent today.
 6. **`retry.otherwise` cycle / self-reference** — `applyRetry` terminates at
    runtime via its `visited` guard, but the guard's "accept the repeated target
@@ -400,7 +400,7 @@ the definition) — they fit `PatternMachine.step`'s purity discipline.
 - **Self-explaining refusals.** A no-match refusal already lists the state's
   declared patterns; for a user debugging their own machine, also list the
   pending changes that failed to match (status + path, first N). "Your tree has
-  `M src/x.ts` but this state only matches `A .gtd/FORMAT.md`" turns the most
+  `M src/x.ts` but this state only matches `A .gtd/FEEDBACK.md`" turns the most
   common custom-workflow dead end from a mystery into a diff-vs-pattern
   diagnosis the user can act on immediately.
 

package/docs/design/steering-file-loops.md

@@ -1,5 +1,16 @@
 # Plan: steering-file validation loops in the default workflow
 
+> SUPERSEDED (§2–§4): the in-machine validation-loop states this plan added —
+> `todo-validating` and `review-validating` — and their `.gtd/FORMAT.md`
+> steering file have all been DELETED. Deterministic format-checking now lives
+> in the `gtd validate` command (which runs the same pure `parseOpenQuestions` /
+> `parseReviewDoc` parsers the state's `mode:` selects) plus producing-agent
+> self-validation, rather than in bash-port `check`/`script` states. The default
+> workflow is now 10 states, not 12. See
+> [docs/design/steering-file-validation-command.md](steering-file-validation-command.md)
+> and [STATES.md §12](../STATES.md). §1 (the file formats) and §5 (the LSP
+> resurrection) below remain valid and current.
+
 > Status: §2–§4 (the two validation loops in the default workflow) AND §5 (the
 > LSP resurrection) both LANDED 2026-07-22. Goal: the bundled default workflow
 > completely maps the functionality the deleted v2 LSP server (`src/Lsp.ts`,