gcf-harness-aif-ext 0.9.2 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +118 -114
- package/commands/harness.js +7 -0
- package/commands/metrics.js +144 -0
- package/commands/run.js +89 -8
- package/extension.json +1 -1
- package/hooks/implement-guard.js +44 -5
- package/hooks/metrics.js +41 -0
- package/hooks/readset-guard.js +6 -1
- package/package.json +1 -1
- package/skills/gcf-run/SKILL.md +80 -80
package/README.md
CHANGED
|
@@ -1,114 +1,118 @@
|
|
|
1
|
-
# gcf-harness-aif-ext
|
|
2
|
-
|
|
3
|
-
Claude-Code-specific orchestration for the [Game Context Framework](../gcf-aif-ext). Optional: GCF works without it, by hand.
|
|
4
|
-
|
|
5
|
-
> **Claude Code only.** Codex and the other AI Factory runtimes are not supported: the harness is built on Claude Code
|
|
6
|
-
> dynamic workflows and `PreToolUse` hooks, which they do not have. AI Factory still copies the `gcf-run` skills into
|
|
7
|
-
> every runtime the project is configured for, where they do not work — on a project without Claude Code, do not
|
|
8
|
-
> install the harness. GCF itself keeps working on any runtime.
|
|
9
|
-
|
|
10
|
-
The harness turns policy-as-text into an executable pipeline. It has **no memory of its own** — everything it knows between two agent calls is on disk in `gcf/`, so a run killed at any point restarts and reaches the same state.
|
|
11
|
-
|
|
12
|
-
## Requirements
|
|
13
|
-
|
|
14
|
-
- [`gcf-aif-ext`](../gcf-aif-ext) `^0.7.0` installed, with a `gcf/` tree in the project — declared as a peer dependency, and `gcf-harness init` refuses when the `gcf` command is not registered
|
|
15
|
-
- Claude Code 2.1.270+ (hooks, dynamic workflows) — the only supported runtime
|
|
16
|
-
- AI Factory 2.x, Node.js ≥ 22.14
|
|
17
|
-
|
|
18
|
-
## Install
|
|
19
|
-
|
|
20
|
-
The shortest way is from the core: `ai-factory gcf init` asks whether to install the harness too, and on yes runs both steps below. It asks only in a terminal and only when the project has the Claude Code agent; `gcf init --harness` installs it without asking, `--no-harness` skips the question.
|
|
21
|
-
|
|
22
|
-
By hand, into a project that already has the core, from npm or a local checkout:
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
ai-factory extension add gcf-aif-ext # the core, if not yet installed
|
|
26
|
-
ai-factory extension add gcf-harness-aif-ext # npm
|
|
27
|
-
# ai-factory extension add ../GCF/packages/gcf-harness-aif-ext # local checkout
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
This installs the `gcf-run` and `gcf-run-ingest` skills and registers the `gcf-harness` command. Then initialize the harness in the project — like `gcf init`, once per project:
|
|
31
|
-
|
|
32
|
-
```bash
|
|
33
|
-
ai-factory gcf-harness init # per project: .claude/settings.json + .gitignore
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
It is **per project**, not global — run it again in every game repository: it adds the readset-guard and implement-guard entries to the project's `.claude/settings.json` (never `~/.claude`) and `.gcf-session/` to `.gitignore`. It also allows what an unattended `claude -p` run needs at least — `Edit`, `Write`, `Bash(git *)`, `Bash(ai-factory *)`, and the same two as `PowerShell(…)` rules, the tool the agent runs commands with on Windows — because print mode cannot ask and denies the rest; add your engine's tools yourself (for example `mcp__<editor-server>` for an editor driven over MCP). Commit `.claude/settings.json`: the rules apply to every Claude session in the project. It is idempotent — a re-run after moving or cloning the project repoints the hook at the new path — refuses when the `gcf` command is not registered, and refuses to rewrite a `settings.json` it cannot parse. It is a separate step for the same reason as `gcf hooks-install`: an AI Factory manifest cannot install Claude Code hooks.
|
|
37
|
-
|
|
38
|
-
## Upgrade
|
|
39
|
-
|
|
40
|
-
```bash
|
|
41
|
-
ai-factory extension update gcf-harness-aif-ext
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
Updating keeps the hook entry valid — it points into the extension's install directory, which an update refreshes in place.
|
|
45
|
-
|
|
46
|
-
0.5.0 drives work items (`gcf/work/<id>.md`, schema 3) instead of cards and requires `gcf-aif-ext` 0.4.0 — update both together; against an older core the gate refuses with `gcf-aif-ext >= 0.4.0 is required`. A project still holding schema 2 cards in `gcf/cards/` re-creates them first (see the core's "Upgrading from 0.3.x"). What changes for you: `/gcf-run` takes ids of items in `ready`; it no longer stops at `built` but ends each item with `submit`, `close` or `block`; the plan is improved in passes until it converges; the WIP cap counts only items that end waiting for a verdict.
|
|
47
|
-
|
|
48
|
-
0.5.1 is a fix release. During a run, `gcf/config.yml` is always readable, and a Grep inside `gcf/` reads only readset files. An import breaks a `depends:` cycle the pair checks closed and asks a D1 about it. It cuts a section longer than a segment at paragraphs, and lens and pair prompts carry a window around the quote. `/gcf-run` checks the plan against each pillar's invariant and asks each pillar's review question over the diff. Use it with core 0.4.1, which brings the parent feature into the readset.
|
|
49
|
-
|
|
50
|
-
0.9.
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
- `/gcf-run
|
|
54
|
-
-
|
|
55
|
-
-
|
|
56
|
-
|
|
57
|
-
0.
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
- A
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
- `
|
|
74
|
-
-
|
|
75
|
-
- `
|
|
76
|
-
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
0.
|
|
81
|
-
|
|
82
|
-
0.6.
|
|
83
|
-
|
|
84
|
-
0.
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
**`ai-factory gcf-harness
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
##
|
|
113
|
-
|
|
114
|
-
|
|
1
|
+
# gcf-harness-aif-ext
|
|
2
|
+
|
|
3
|
+
Claude-Code-specific orchestration for the [Game Context Framework](../gcf-aif-ext). Optional: GCF works without it, by hand.
|
|
4
|
+
|
|
5
|
+
> **Claude Code only.** Codex and the other AI Factory runtimes are not supported: the harness is built on Claude Code
|
|
6
|
+
> dynamic workflows and `PreToolUse` hooks, which they do not have. AI Factory still copies the `gcf-run` skills into
|
|
7
|
+
> every runtime the project is configured for, where they do not work — on a project without Claude Code, do not
|
|
8
|
+
> install the harness. GCF itself keeps working on any runtime.
|
|
9
|
+
|
|
10
|
+
The harness turns policy-as-text into an executable pipeline. It has **no memory of its own** — everything it knows between two agent calls is on disk in `gcf/`, so a run killed at any point restarts and reaches the same state.
|
|
11
|
+
|
|
12
|
+
## Requirements
|
|
13
|
+
|
|
14
|
+
- [`gcf-aif-ext`](../gcf-aif-ext) `^0.7.0` installed, with a `gcf/` tree in the project — declared as a peer dependency, and `gcf-harness init` refuses when the `gcf` command is not registered
|
|
15
|
+
- Claude Code 2.1.270+ (hooks, dynamic workflows) — the only supported runtime
|
|
16
|
+
- AI Factory 2.x, Node.js ≥ 22.14
|
|
17
|
+
|
|
18
|
+
## Install
|
|
19
|
+
|
|
20
|
+
The shortest way is from the core: `ai-factory gcf init` asks whether to install the harness too, and on yes runs both steps below. It asks only in a terminal and only when the project has the Claude Code agent; `gcf init --harness` installs it without asking, `--no-harness` skips the question.
|
|
21
|
+
|
|
22
|
+
By hand, into a project that already has the core, from npm or a local checkout:
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
ai-factory extension add gcf-aif-ext # the core, if not yet installed
|
|
26
|
+
ai-factory extension add gcf-harness-aif-ext # npm
|
|
27
|
+
# ai-factory extension add ../GCF/packages/gcf-harness-aif-ext # local checkout
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
This installs the `gcf-run` and `gcf-run-ingest` skills and registers the `gcf-harness` command. Then initialize the harness in the project — like `gcf init`, once per project:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
ai-factory gcf-harness init # per project: .claude/settings.json + .gitignore
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
It is **per project**, not global — run it again in every game repository: it adds the readset-guard and implement-guard entries to the project's `.claude/settings.json` (never `~/.claude`) and `.gcf-session/` to `.gitignore`. It also allows what an unattended `claude -p` run needs at least — `Edit`, `Write`, `Bash(git *)`, `Bash(ai-factory *)`, and the same two as `PowerShell(…)` rules, the tool the agent runs commands with on Windows — because print mode cannot ask and denies the rest; add your engine's tools yourself (for example `mcp__<editor-server>` for an editor driven over MCP). Commit `.claude/settings.json`: the rules apply to every Claude session in the project. It is idempotent — a re-run after moving or cloning the project repoints the hook at the new path — refuses when the `gcf` command is not registered, and refuses to rewrite a `settings.json` it cannot parse. It is a separate step for the same reason as `gcf hooks-install`: an AI Factory manifest cannot install Claude Code hooks.
|
|
37
|
+
|
|
38
|
+
## Upgrade
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
ai-factory extension update gcf-harness-aif-ext
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Updating keeps the hook entry valid — it points into the extension's install directory, which an update refreshes in place.
|
|
45
|
+
|
|
46
|
+
0.5.0 drives work items (`gcf/work/<id>.md`, schema 3) instead of cards and requires `gcf-aif-ext` 0.4.0 — update both together; against an older core the gate refuses with `gcf-aif-ext >= 0.4.0 is required`. A project still holding schema 2 cards in `gcf/cards/` re-creates them first (see the core's "Upgrading from 0.3.x"). What changes for you: `/gcf-run` takes ids of items in `ready`; it no longer stops at `built` but ends each item with `submit`, `close` or `block`; the plan is improved in passes until it converges; the WIP cap counts only items that end waiting for a verdict.
|
|
47
|
+
|
|
48
|
+
0.5.1 is a fix release. During a run, `gcf/config.yml` is always readable, and a Grep inside `gcf/` reads only readset files. An import breaks a `depends:` cycle the pair checks closed and asks a D1 about it. It cuts a section longer than a segment at paragraphs, and lens and pair prompts carry a window around the quote. `/gcf-run` checks the plan against each pillar's invariant and asks each pillar's review question over the diff. Use it with core 0.4.1, which brings the parent feature into the readset.
|
|
49
|
+
|
|
50
|
+
0.9.4 keeps run metrics (Harness-PRD §11) and adds `ai-factory gcf-harness metrics`. Use it with core 0.7.13, which logs the plan checks and the DoD attempts, and refuses `close` and `submit` without a green `gcf verify` of the code as it is. `/gcf-run` runs the verify again after a review fix; implement-guard refuses `gcf close` or `gcf submit` with `--unverified` in a `/gcf-run` session, since a move past the DoD gate is the human's call. A main-mode item whose move went past it anyway (the commit says `(unverified: …)`) reads `— moved without a green gcf verify` on its report line, and `gcf-harness metrics` counts such moves.
|
|
51
|
+
|
|
52
|
+
0.9.2 holds `/gcf-run` to aif-implement. Run `ai-factory gcf-harness init` again after the update: it registers the new hook.
|
|
53
|
+
- The implement guard: in a session that runs `/gcf-run`, a write to a code file is denied until the session invokes the aif-implement skill (gcf-run step 13), and so is a plan tick, which only aif-implement sets. Planning stays free: `gcf/`, `.ai-factory/`, `.gcf-session/` and any markdown file. After aif-implement every write is free, the fixes of a red DoD and of the review included. It works in the harness's runs and in an interactive `/gcf-run` alike, and touches no other session.
|
|
54
|
+
- `/gcf-run` converges a plan when an improve pass finds nothing blocking: a wrong or missing behavior or task, a broken pillar invariant, a rule conflict, a task no check can prove. Polish (wording, splitting tasks, order) is applied, but no longer keeps the loop going. Before, a pass that changed any task asked for one more, and a fresh reviewer almost always changes something. The report row per pass gives the blocking and polish counts.
|
|
55
|
+
- `/gcf-run` answers aif-plan's own questions (tests, logging, docs) itself, from the item, its DoD and the project's rules.
|
|
56
|
+
- The run report marks an item whose session wrote code without aif-implement: `— code written outside aif-implement` on its line, `process` in `--json` and in its kept record. It reads the session's own events, its subagents' included, so it also covers a project without the guard registered. A write through Bash is seen by neither: the review of the diff is the check there.
|
|
57
|
+
- Lessons (with core 0.7.10): after each agent session the run asks the same session, resumed, what got in its way - tools, rules, the framework, the project template - with the questions of `ai-factory gcf lessons prompt`. Its answers go to the inbox on main as lessons (`gcf lessons add`), and the board shows them on the card, in the drawer and in the inbox. The run passes `--no-lessons` to `/gcf-run`, so the agent carries no lessons instruction while it works. The turn follows a result, a handoff (before the fresh session), a timeout or an error; not a usage limit or a stop, and only for a `claude` agent with a session id. Another agent asks its lessons itself at the end of a run on main; in a worktree it records none. It takes up to 10 minutes; its cost goes on the item, and the report line says `, N lessons`. A failed lessons turn is logged and changes nothing else. A project that turns lessons off, or a core before 0.7.10, gets no turn and no flag.
|
|
58
|
+
|
|
59
|
+
0.9.1 is a fix release:
|
|
60
|
+
- A stopped session reads as `no result (stopped)`, and its kept record as `ended: stopped`: before, the kill's exit code (`agent exit 1` on Windows).
|
|
61
|
+
- `gcf-harness stop <id>` never kills the run: a stop before the item's session starts (or between two sessions of a handoff) leaves `.gcf-session/runs/<id>.stop`, and that session is stopped as it starts. Before, a stop then killed the whole run with the other items of its batch. A run hung before its first session (in its worktree setup) is not stopped. On main, a stop marks the item too once the run has reached it. Known limit: a stop in the moment between a worktree run's claim of the item and its clearing of an old mark is lost.
|
|
62
|
+
- A stop that comes while the session starts no longer misses it.
|
|
63
|
+
|
|
64
|
+
0.9.0, to update together with core 0.7.6:
|
|
65
|
+
- The harness's own agent runs with `--permission-mode bypassPermissions`: it may do everything but the project's deny rules, which hold in every mode. The rules come from the deny groups of `gcf/config.yml` (`ai-factory gcf permissions`, or the board): git history and uncommitted work, recursive delete, publishing, the human's GCF moves and the guards are on by default; secrets, changes outside the project and killing processes are off. `git push` is allowed. Deny rules match the command as written: they catch accidents, not a rewording, and `/gcf-run` makes a refused command a stop with `needs permission: <command>`.
|
|
66
|
+
- A run loads the project's settings and MCP servers only (`--setting-sources project,local --strict-mcp-config --mcp-config .mcp.json`): the operator's plugins, hooks and claude.ai connectors stay out. So does `env` or `apiKeyHelper` in `~/.claude/settings.json`: an operator who logs in through them, not `claude login`, names the agent with `--agent`. The deny rules and the handoff guard come in `.gcf-session/runs/agent-settings.json` (`--settings`), so a worktree gets them without a commit.
|
|
67
|
+
- Context handoff: past `context_handoff_pct` (60 by default) of the window, or at a compaction, the run flags the session; the handoff guard then denies every call of the main agent but the plan's read and edit with the instruction to write `## Handoff` into the plan and end. The run starts a fresh session with `/gcf-run <id> --handoff`, which goes on from the plan: the ticked tasks stay done. A session that works on past its flag is stopped; after 3 handoffs in a row the item is blocked with `context handoff limit`. The window is `context_window`, else the one a result of the same model reported (`.gcf-session/runs/windows.json`); with neither, the first session of a model runs without a handoff and says so. The sessions of an item share one timeout, and their costs add up; `--json` gives `handoffs`.
|
|
68
|
+
- `auto_compact_pct` sets the agent's `CLAUDE_AUTOCOMPACT_PCT_OVERRIDE`, at least 15 points above the handoff share.
|
|
69
|
+
- The readset-guard reads the manifest of the directory the session started in (`CLAUDE_PROJECT_DIR`): an agent's `cd` no longer lifts the readset.
|
|
70
|
+
- `/gcf-run` writes a plan task's code through aif-implement, so the plan ticks show the item's progress, and answers aif-implement's questions itself: no commit, stash or merge.
|
|
71
|
+
|
|
72
|
+
0.8.0, to update together with core 0.7.0:
|
|
73
|
+
- `run_mode: main` in `gcf/config.yml` (or `run --mode main`): items run one after another in the main checkout, for an agent that works through an editor bound to the project folder. The item carries the core's session mark (`gcf claim`), not a run lock, so the agent records its own move; one run works the checkout at a time (`.gcf-session/runs/.main-run.lock`).
|
|
74
|
+
- `gcf-harness next [N] --json`: the ready items that can run together now, by priority; each is taken while the batch with it passes the gate.
|
|
75
|
+
- `gcf-harness stop <id>`: stops the session that works the item (its run lock names it); the run records no result for it and goes on with the rest. The item stays ready and its work is kept. Known limit: the session is killed at once, so a `gcf` move it was in the middle of may leave `.gcf-session/main.lock` behind (taken over once its pid is gone).
|
|
76
|
+
- A run on main refreshes its item's mark every 5 minutes, and the readset-guard hook refreshes live marks on each read: a long build without a gcf command does not expire it.
|
|
77
|
+
- `run` refuses an id that is not a slug before it names a lock file or builds the agent's command line.
|
|
78
|
+
- The gate lets a session through for the item its own run marked (`GCF_RUN_ITEM`); `/gcf-run` outside a worktree claims each item after its gate and releases one it leaves without a move.
|
|
79
|
+
|
|
80
|
+
0.7.0, to update together with core 0.6.0: an item's `model:` and `effort:` (`gcf item new --model <m> --effort <e>`) go to its agent as `--model <m> --effort <e>`, read from `gcf board --json`; a value that is not one plain word fails that item and starts no session. `--timeout <minutes>` (120 by default) stops a session that runs longer, with every process it started; its item stays ready, and its run lock is freed. The gate counts the items other runs drive now: their places under the WIP limit, and their systems — a board starts one run per click. Requires `gcf-aif-ext` 0.6.
|
|
81
|
+
|
|
82
|
+
0.6.1 is a fix release, to update together with core 0.5.3. The run lock `.gcf-session/runs/<id>.lock` is created exclusively, and a lock left by a dead run is taken over; while it is live, the gate refuses the item to another run and the core refuses any move on it, `/gcf-run` on main included. A session whose log cannot open ends as an error and releases its claim.
|
|
83
|
+
|
|
84
|
+
0.6.0 adds `ai-factory gcf-harness run` (a batch at once, a worktree and an agent session per item, then the integrator) and the `--worktree` mode of `/gcf-run`, whose session ritual now integrates the verified branches a dead run left. It requires `gcf-aif-ext` 0.5.0 (`gcf worktree`, `gcf integrate`, `gcf verify`) — update both together and run `ai-factory gcf init`. The DoD of `/gcf-run` is `ai-factory gcf verify <id>`, and the item's `live:` delta is written before it.
|
|
85
|
+
|
|
86
|
+
0.4.1 is a patch release. `/gcf-run` now commits the implementation first, then flips the card to `built` in a second commit, with `built_at` set to the implementation commit's hash. Update the core to 0.3.1 together with the harness: without it, an audit run before a verdict makes the pre-commit gate refuse that verdict.
|
|
87
|
+
|
|
88
|
+
To remove the extension:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
ai-factory extension remove gcf-harness-aif-ext
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Removal does not touch `.claude/settings.json`: delete the entries whose command ends in `# gcf-readset-guard` or `# gcf-implement-guard` (under `PreToolUse` and `UserPromptSubmit`) by hand, or the tools they match will report a hook error.
|
|
95
|
+
|
|
96
|
+
## What it adds
|
|
97
|
+
|
|
98
|
+
**`/gcf-run <item-id…>`** — drives work items through plan, implementation and DoD, with every gate checked *before* the tokens are spent: the audit ritual, the gate below, the context budget. The plan is written by `aif-plan`, then improved in passes, each by a fresh subagent that reads only the plan and the readset and records every default it picks; `ai-factory gcf plan check <id>` runs after each pass, and a pass with no blocking finding (a wrong or missing behavior or task, a broken pillar invariant, a rule conflict, an unprovable task) ends the loop. A blocking finding left after `plan_passes` passes (3 by default), or a question only the human can answer (a save format, a public API, a new dependency, a pillar conflict), blocks the item before any code is written. The DoD is `ai-factory gcf verify <id>` — the gate's checks, the item's `live:` delta, and the project's `dod.build` and `dod.test` commands from `gcf/config.yml` — plus a review by a fresh subagent, up to `dod_attempts` attempts. Every item ends in a state the core records: `gcf submit` (gate `verdict` or `decision` — waits for the human), `gcf close` (gate `auto` — done, or a soft unit closed as `unplayed`) or `gcf block` with the reason. The verdict and the decision belong to the human: the pipeline stops at `submit` and prints the playtest list.
|
|
99
|
+
|
|
100
|
+
**`ai-factory gcf-harness gate <item-id…>`** — the `/gcf-run` gate as code. It reads one `gcf audit --json` report and exits 1 with a one-line reason when the items may not go: a core older than 0.4.0, a `schema:` other than 1 in `gcf/config.yml`, an id with no work item, a feature (run its children; the feature closes when they are finished), an item that is not `ready`, an unplayed dependency, no WIP room for the batch (items awaiting a verdict + the batch's `gate: verdict` items over `wip_built_limit`), or two items of the batch touching the same system unless both say `parallel: ok`. The tune limit is not a gate rule: `verdict tune` itself refuses at the limit, and an item tuned up to it still gets its last build. The skill runs it and stops on a non-zero exit — no model compares the numbers.
|
|
101
|
+
|
|
102
|
+
**`ai-factory gcf-harness run <item-id…> [--agent "<cmd>"]`** — runs a batch at once, each item in its own worktree (`gcf worktree <id>`) with its own agent session: `claude -p --output-format stream-json --verbose "/gcf-run <id> --worktree"` by default, or the command in `--agent` / `GCF_RUN_AGENT`. The sessions run unattended, so set the agent's permissions for print mode first. A session ends with a commit and `gcf verify <id> --mark` (the `GCF-Verified` trailer); a question for the human goes into `.gcf-session/outcome.json` in its worktree, and `run` records the block on main. Then `gcf integrate` merges the verified branches and records each move, and `run` prints the playtest session. Its first step, like `/gcf-run`'s, integrates the verified branches a dead run left. A session that ended with no result leaves its item `ready`; the next `run` resumes its worktree and the agent's own session (`--resume <session_id>`, kept in `.gcf-session/runs/<id>.agent.json`), in either run mode; `--fresh` starts a new session instead. A session cut off by Claude's usage limit reports `paused (usage limit, resets 3am (…))` and `limit_resets`. Logs: `.gcf-session/runs/<id>.log`, written as the session works: a `start:` line, then one stamped line per agent message, tool call (`tool: Edit <file>`, `tool: Bash <command>`) and failed tool result, and at the end the agent's final text and `done in 34m, 212 turns, $3.41 (…tokens)`; stderr goes in as it is, and a non-Claude `--agent` keeps its own lines. With `--json`, each item carries the session's `cost_usd`, `tokens`, `turns` and `duration_ms` when the agent reported them. One run drives an item at a time (`.gcf-session/runs/<id>.lock`), and a resumed worktree first gets main merged in, so the answer from an `unblock` reaches the session. A worktree sees only what is committed: keep `.ai-factory.json`, the installed extensions and `.claude/` (skills, the readset-guard hook) in git, or the session has no `/gcf-run` and no `ai-factory gcf`.
|
|
103
|
+
|
|
104
|
+
**`ai-factory gcf-harness metrics [--since <date>] [--json]`** — the run metrics of Harness-PRD §11 from `gcf/.metrics`: how many item runs reached `submit` or `close` (a run the usage limit paused or a session cut short does not count as finished), the block reasons, what the runs cost (dollars, minutes, turns, handoffs, code written outside aif-implement), the plan checks per item (the improve passes), the DoD attempts per item and the checks that failed them, and the guard firings by hook and by item. The logs: `run-log.csv` (one line per item of each `run`), `guard-log.csv` (each denial of readset-guard and implement-guard, by hook: `readset`, `implement`, or `verify` for a move past the verify gate), and the core's `plan-log.csv` and `verify-log.csv`. A worktree session's lines go to the main checkout's logs; the next `gcf integrate` commits them.
|
|
105
|
+
|
|
106
|
+
**`/gcf-run-ingest <gdd.md>`** — imports a design document as a fan-out workflow: parallel aspect extraction, provenance verified by code rather than by an agent (a quote either is a verbatim substring of its section or it is not), completeness lenses per candidate, and an interaction matrix computed from the candidates that write the same state.
|
|
107
|
+
|
|
108
|
+
**readset-guard** — a `PreToolUse` hook on Read/Grep/Glob. While a run manifest exists at `.gcf-session/readset.json`, a path inside `gcf/` that is not in the readset is denied, and so is a Grep from the project root (no `path`, or a `path` above `gcf/`) unless its `glob` starts with a literal directory other than `gcf` (`src/**`, not `**/*.md`), because such a content search would read the whole tree past the gate. Glob and Greps scoped to the game code are never restricted. No manifest means no restriction: manual work is never gated. An honest limit — `cat` through Bash still bypasses it. This is a discipline tool, not a security perimeter.
|
|
109
|
+
|
|
110
|
+
**implement-guard** — a `UserPromptSubmit` and `PreToolUse` hook (Skill, the write tools, Bash and PowerShell). A session that starts `/gcf-run` (typed, passed to `claude -p`, or invoked as a skill) gets a state in `.gcf-session/guard/<session_id>.json`; until it invokes the aif-implement skill, Edit, Write, MultiEdit and NotebookEdit are denied outside `gcf/`, `.ai-factory/`, `.gcf-session/` and markdown, and a markdown write that adds a `- [x]` tick is denied too. Any other session is never touched, and an error lets the call through. A `gcf submit` or `gcf close` with `--unverified` is refused in such a session, after its end move too (a close the core refused does not free it for a skip): only the human moves an item past the DoD gate, from a terminal of their own. Such a refusal is logged with the hook name `verify`. The move that ends the item (`gcf block`, `submit` or `close`, through Bash or PowerShell) lets the session go, so the operator's own edits after a blocked run are free again. A plugin's skill (`<plugin>:aif-implement`) counts too. Limits: the flag is per session, so in a batch the first item's aif-implement frees the rest; a write through Bash is not seen, by the run report either.
|
|
111
|
+
|
|
112
|
+
## Docs
|
|
113
|
+
|
|
114
|
+
[Harness-PRD](https://github.com/VirtualMaestro/GCF/blob/main/docs/Harness-PRD.md) is the spec: pipeline stages, gates, execution surfaces, the guard's contract.
|
|
115
|
+
|
|
116
|
+
## License
|
|
117
|
+
|
|
118
|
+
MIT
|
package/commands/harness.js
CHANGED
|
@@ -8,6 +8,7 @@ import { existsSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
|
|
|
8
8
|
import { spawnSync } from 'node:child_process';
|
|
9
9
|
import path from 'node:path';
|
|
10
10
|
import { fileURLToPath } from 'node:url';
|
|
11
|
+
import { metricsCommand } from './metrics.js';
|
|
11
12
|
import { run, cliCore, DEFAULT_AGENT, DEFAULT_TIMEOUT_MIN, MAIN_RUN, stopOf } from './run.js';
|
|
12
13
|
|
|
13
14
|
const EXT_ROOT = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
|
|
@@ -374,6 +375,12 @@ export function register(program) {
|
|
|
374
375
|
const ids = nextBatch(auditReport(), Math.max(1, Number(count ?? 1) || 1));
|
|
375
376
|
console.log(o.json ? JSON.stringify(ids) : ids.join('\n') || 'nothing can run now');
|
|
376
377
|
});
|
|
378
|
+
harness
|
|
379
|
+
.command('metrics')
|
|
380
|
+
.description('Run metrics from gcf/.metrics: reached moves, cost, plan checks, DoD, guards')
|
|
381
|
+
.option('--since <date>', 'only lines at or after this ISO date or time')
|
|
382
|
+
.option('--json', 'machine output')
|
|
383
|
+
.action(metricsCommand);
|
|
377
384
|
harness
|
|
378
385
|
.command('stop <id>')
|
|
379
386
|
.description('Stop the session that works an item; the run goes on, the item stays ready')
|
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
// `ai-factory gcf-harness metrics`: the run metrics of Harness-PRD §11 from the logs in
|
|
2
|
+
// gcf/.metrics — runs that reached the core's move, what they cost, the plan checks and DoD
|
|
3
|
+
// attempts they took, the guard firings. A view of the logs: it stores nothing.
|
|
4
|
+
import { existsSync, readFileSync } from 'node:fs';
|
|
5
|
+
import path from 'node:path';
|
|
6
|
+
import { metricsDir } from '../hooks/metrics.js';
|
|
7
|
+
|
|
8
|
+
/** The rows of a log at or after `since` (an ISO date or time); none when it is not there. */
|
|
9
|
+
function rows(dir, file, since) {
|
|
10
|
+
const f = path.join(dir, file);
|
|
11
|
+
if (!existsSync(f)) return [];
|
|
12
|
+
|
|
13
|
+
return readFileSync(f, 'utf8')
|
|
14
|
+
.split(/\r?\n/)
|
|
15
|
+
.filter(Boolean)
|
|
16
|
+
.map((l) => l.split(','))
|
|
17
|
+
.filter((r) => !since || r[0] >= since);
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
const num = (v) => (v === '' || v === undefined ? null : Number(v));
|
|
21
|
+
const sum = (xs) => xs.reduce((a, b) => a + b, 0);
|
|
22
|
+
const avg = (xs) => (xs.length ? sum(xs) / xs.length : null);
|
|
23
|
+
|
|
24
|
+
/** `{key: count}` of a list, the most frequent first, at most `top` keys. */
|
|
25
|
+
function counts(xs, top = Infinity) {
|
|
26
|
+
const m = new Map();
|
|
27
|
+
for (const x of xs) m.set(x, (m.get(x) ?? 0) + 1);
|
|
28
|
+
|
|
29
|
+
return Object.fromEntries([...m].sort((a, b) => b[1] - a[1]).slice(0, top));
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
/** How many rows each item has: the checks or attempts per item. */
|
|
33
|
+
const perItem = (rs) => Object.values(counts(rs.map((r) => r[1])));
|
|
34
|
+
|
|
35
|
+
// A run ended by the usage limit or cut short (timeout, stop, crash) did not finish: its item
|
|
36
|
+
// goes on in the next run, and the share counts only the runs that did. An item refused before
|
|
37
|
+
// its session (`not-started`: another run holds it, its worktree failed) never ran at all.
|
|
38
|
+
const UNFINISHED = ['paused', 'no-result', 'not-started'];
|
|
39
|
+
|
|
40
|
+
/** The summary of the logs in `dir`, from `since` on. */
|
|
41
|
+
export function summarize(dir, since = '') {
|
|
42
|
+
// run-log: at, item, mode, result, reason, cost_usd, input, output, cache_read, turns,
|
|
43
|
+
// duration_ms, handoffs, process
|
|
44
|
+
const runs = rows(dir, 'run-log.csv', since);
|
|
45
|
+
const finished = runs.filter((r) => !UNFINISHED.includes(r[3]));
|
|
46
|
+
const reached = finished.filter((r) => ['submit', 'close'].includes(r[3]));
|
|
47
|
+
const spent = runs.map((r) => num(r[5])).filter((v) => v !== null);
|
|
48
|
+
// plan-log: at, item, ok, problems; verify-log: at, item, ok, failed checks, mark
|
|
49
|
+
const plans = rows(dir, 'plan-log.csv', since);
|
|
50
|
+
const verifies = rows(dir, 'verify-log.csv', since).filter((r) => r[4] !== '1');
|
|
51
|
+
// guard-log: at, item, hook, tool
|
|
52
|
+
const guards = rows(dir, 'guard-log.csv', since);
|
|
53
|
+
|
|
54
|
+
return {
|
|
55
|
+
since: since || null,
|
|
56
|
+
runs: {
|
|
57
|
+
count: runs.length,
|
|
58
|
+
results: counts(runs.map((r) => r[3])),
|
|
59
|
+
finished: finished.length,
|
|
60
|
+
reached: reached.length,
|
|
61
|
+
share: finished.length ? reached.length / finished.length : null,
|
|
62
|
+
blocks: counts(
|
|
63
|
+
runs.filter((r) => r[3] === 'block').map((r) => r[4] || '(no reason)'),
|
|
64
|
+
5,
|
|
65
|
+
),
|
|
66
|
+
cost_usd: sum(spent),
|
|
67
|
+
avg_cost_usd: avg(spent),
|
|
68
|
+
avg_duration_ms: avg(runs.map((r) => num(r[10])).filter((v) => v !== null)),
|
|
69
|
+
avg_turns: avg(runs.map((r) => num(r[9])).filter((v) => v !== null)),
|
|
70
|
+
handoffs: sum(runs.map((r) => num(r[11]) ?? 0)),
|
|
71
|
+
outside_implement: runs.filter((r) => r[12]?.includes('outside aif-implement')).length,
|
|
72
|
+
unverified: runs.filter((r) => r[12]?.includes('without a green gcf verify')).length,
|
|
73
|
+
},
|
|
74
|
+
plan: {
|
|
75
|
+
items: perItem(plans).length,
|
|
76
|
+
checks: plans.length,
|
|
77
|
+
failed: plans.filter((r) => r[2] !== '1').length,
|
|
78
|
+
avg_checks: avg(perItem(plans)),
|
|
79
|
+
},
|
|
80
|
+
verify: {
|
|
81
|
+
items: perItem(verifies).length,
|
|
82
|
+
attempts: verifies.length,
|
|
83
|
+
red: verifies.filter((r) => r[2] !== '1').length,
|
|
84
|
+
avg_attempts: avg(perItem(verifies)),
|
|
85
|
+
failing: counts(verifies.flatMap((r) => (r[3] ? r[3].split(' ') : []))),
|
|
86
|
+
},
|
|
87
|
+
guard: {
|
|
88
|
+
count: guards.length,
|
|
89
|
+
by_hook: counts(guards.map((r) => r[2])),
|
|
90
|
+
by_item: counts(
|
|
91
|
+
guards.map((r) => r[1] || '(no item)'),
|
|
92
|
+
5,
|
|
93
|
+
),
|
|
94
|
+
},
|
|
95
|
+
};
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
const list = (o) =>
|
|
99
|
+
Object.entries(o)
|
|
100
|
+
.map(([k, n]) => `${k} ${n}`)
|
|
101
|
+
.join(', ');
|
|
102
|
+
const one = (v, digits = 1) => (v === null ? '-' : v.toFixed(digits));
|
|
103
|
+
|
|
104
|
+
/** The summary as an operator reads it. */
|
|
105
|
+
export function formatMetrics(s) {
|
|
106
|
+
const r = s.runs;
|
|
107
|
+
const share = r.share === null ? '' : ` (${Math.round(r.share * 100)}%)`;
|
|
108
|
+
const failing = list(s.verify.failing);
|
|
109
|
+
const minutes = r.avg_duration_ms === null ? '-' : one(r.avg_duration_ms / 60_000, 0);
|
|
110
|
+
|
|
111
|
+
return [
|
|
112
|
+
...(s.since ? [`since ${s.since}`] : []),
|
|
113
|
+
`runs: ${r.count} item runs (${list(r.results) || 'none'}); ` +
|
|
114
|
+
`${r.reached}/${r.finished} finished ones reached submit or close${share}`,
|
|
115
|
+
...(Object.keys(r.blocks).length
|
|
116
|
+
? [
|
|
117
|
+
`blocks: ${Object.entries(r.blocks)
|
|
118
|
+
.map(([why, n]) => `${why} ×${n}`)
|
|
119
|
+
.join('; ')}`,
|
|
120
|
+
]
|
|
121
|
+
: []),
|
|
122
|
+
`cost: $${r.cost_usd.toFixed(2)}, $${one(r.avg_cost_usd, 2)} a run, ${minutes} min, ` +
|
|
123
|
+
`${one(r.avg_turns, 0)} turns; ${r.handoffs} handoffs; ` +
|
|
124
|
+
`${r.outside_implement} with code outside aif-implement, ` +
|
|
125
|
+
`${r.unverified} moved without a green verify`,
|
|
126
|
+
`plan: ${s.plan.items} items, ${one(s.plan.avg_checks)} checks each, ${s.plan.failed} failed`,
|
|
127
|
+
`verify: ${s.verify.items} items, ${one(s.verify.avg_attempts)} attempts each, ` +
|
|
128
|
+
`${s.verify.red} red${failing ? ` (${failing})` : ''}`,
|
|
129
|
+
`guard: ${s.guard.count} firings` +
|
|
130
|
+
(s.guard.count ? ` (${list(s.guard.by_hook)}); most: ${list(s.guard.by_item)}` : ''),
|
|
131
|
+
].join('\n');
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
/** The `metrics` command: the project's logs, from `--since` on. */
|
|
135
|
+
export function metricsCommand(o) {
|
|
136
|
+
// The logs stamp UTC: a local date or time is turned into one before the comparison. A bare
|
|
137
|
+
// date is local midnight (JavaScript would read it as UTC).
|
|
138
|
+
const local = /^\d{4}-\d\d-\d\d$/.test(o.since ?? '') ? `${o.since}T00:00` : o.since;
|
|
139
|
+
const at = o.since ? new Date(local) : null;
|
|
140
|
+
if (at && Number.isNaN(at.getTime()))
|
|
141
|
+
throw new Error(`--since ${o.since} is not a date (YYYY-MM-DD or an ISO time)`);
|
|
142
|
+
const s = summarize(metricsDir(process.cwd()), at ? at.toISOString() : '');
|
|
143
|
+
console.log(o.json ? JSON.stringify(s, null, 2) : formatMetrics(s));
|
|
144
|
+
}
|
package/commands/run.js
CHANGED
|
@@ -9,6 +9,7 @@ import path from 'node:path';
|
|
|
9
9
|
import { fileURLToPath } from 'node:url';
|
|
10
10
|
import { activity, clock, count } from './activity.js';
|
|
11
11
|
import { planning, readState } from '../hooks/implement-guard.js';
|
|
12
|
+
import { logMetrics } from '../hooks/metrics.js';
|
|
12
13
|
|
|
13
14
|
// The event stream, not the final answer alone: the log shows the session at work as it goes.
|
|
14
15
|
export const DEFAULT_AGENT = 'claude -p --output-format stream-json --verbose';
|
|
@@ -778,6 +779,8 @@ function resumeLost(rec, ran) {
|
|
|
778
779
|
|
|
779
780
|
/** The report note of an item whose session skipped gcf-run step 13. */
|
|
780
781
|
export const OUTSIDE_IMPLEMENT = 'code written outside aif-implement';
|
|
782
|
+
/** The report note of a main-mode item moved with `--unverified` (its commit says so). */
|
|
783
|
+
export const UNVERIFIED_MOVE = 'moved without a green gcf verify';
|
|
781
784
|
|
|
782
785
|
/**
|
|
783
786
|
* Whether a session wrote code with no aif-implement before it: its stream shows none, and
|
|
@@ -1136,6 +1139,7 @@ async function runMain(projectDir, ids, ctx) {
|
|
|
1136
1139
|
id,
|
|
1137
1140
|
result: 'error',
|
|
1138
1141
|
error: 'another run works in the main checkout: wait for it, or stop it',
|
|
1142
|
+
started: false,
|
|
1139
1143
|
}));
|
|
1140
1144
|
|
|
1141
1145
|
const items = [];
|
|
@@ -1152,6 +1156,8 @@ async function runMain(projectDir, ids, ctx) {
|
|
|
1152
1156
|
async function mainItem(projectDir, id, ctx) {
|
|
1153
1157
|
const { core, logs, agentOf } = ctx;
|
|
1154
1158
|
let r;
|
|
1159
|
+
let started = false;
|
|
1160
|
+
const head = headOf(projectDir);
|
|
1155
1161
|
await rm(stopOf(logs, id), { force: true });
|
|
1156
1162
|
// The checkout's lock names the item from now on: a stop before its session marks it.
|
|
1157
1163
|
writeFileSync(path.join(logs, `${MAIN_RUN}.lock`), `${process.pid}\n${id}\n`);
|
|
@@ -1182,6 +1188,7 @@ async function mainItem(projectDir, id, ctx) {
|
|
|
1182
1188
|
done,
|
|
1183
1189
|
lessonsPrompt,
|
|
1184
1190
|
};
|
|
1191
|
+
started = true;
|
|
1185
1192
|
r = await session(s, ctx);
|
|
1186
1193
|
} catch (e) {
|
|
1187
1194
|
r = { id, error: e.message };
|
|
@@ -1192,15 +1199,87 @@ async function mainItem(projectDir, id, ctx) {
|
|
|
1192
1199
|
const item = await mainResult(core, r);
|
|
1193
1200
|
await remember(logs, r, item);
|
|
1194
1201
|
|
|
1195
|
-
|
|
1202
|
+
// The core lets a move with no green verify through only with `--unverified`, which its
|
|
1203
|
+
// commit names: the session went past its DoD gate.
|
|
1204
|
+
const note = started && movedUnverified(projectDir, id, head) ? UNVERIFIED_MOVE : null;
|
|
1205
|
+
|
|
1206
|
+
// An item refused before its session spent nothing: the metrics keep it out of the share (§11).
|
|
1207
|
+
return { ...item, ...r.cost, ...extras(r, note), ...(started ? {} : { started: false }) };
|
|
1208
|
+
}
|
|
1209
|
+
|
|
1210
|
+
/** The checkout's HEAD, or null before its first commit. */
|
|
1211
|
+
function headOf(dir) {
|
|
1212
|
+
const r = spawnSync('git', ['rev-parse', '--verify', '-q', 'HEAD'], {
|
|
1213
|
+
cwd: dir,
|
|
1214
|
+
encoding: 'utf8',
|
|
1215
|
+
});
|
|
1216
|
+
|
|
1217
|
+
return r.status === 0 ? r.stdout.trim() : null;
|
|
1218
|
+
}
|
|
1219
|
+
|
|
1220
|
+
/**
|
|
1221
|
+
* Whether a commit since `head` moved the item with `--unverified`:
|
|
1222
|
+
* `gcf(close): <id> (unverified: …)`.
|
|
1223
|
+
*/
|
|
1224
|
+
export function movedUnverified(dir, id, head) {
|
|
1225
|
+
if (!head) return false;
|
|
1226
|
+
const r = spawnSync('git', ['log', '--format=%s', `${head}..HEAD`], {
|
|
1227
|
+
cwd: dir,
|
|
1228
|
+
encoding: 'utf8',
|
|
1229
|
+
});
|
|
1230
|
+
// `tray` must not match `tray-2`: the id ends at a space (an id is [a-z0-9-] only).
|
|
1231
|
+
const move = new RegExp(`^gcf\\((submit|close)\\): ${id} .*\\(unverified: `);
|
|
1232
|
+
|
|
1233
|
+
return r.status === 0 && r.stdout.split('\n').some((s) => move.test(s));
|
|
1196
1234
|
}
|
|
1197
1235
|
|
|
1198
|
-
/**
|
|
1199
|
-
|
|
1200
|
-
|
|
1201
|
-
|
|
1202
|
-
|
|
1203
|
-
|
|
1236
|
+
/**
|
|
1237
|
+
* An item's result word and why: `paused (usage limit, resets 3am (…)): …` and
|
|
1238
|
+
* `no result (timeout): …` give their cause in the parentheses.
|
|
1239
|
+
*/
|
|
1240
|
+
function outcome(i) {
|
|
1241
|
+
const text = String(i.result);
|
|
1242
|
+
const cut = text.startsWith('no result') ? 'no-result' : text.split(/[ (]/)[0];
|
|
1243
|
+
const word = i.started === false ? 'not-started' : cut;
|
|
1244
|
+
const cause = /^[^(]*\((.*?)\)(?::|$)/.exec(text)?.[1];
|
|
1245
|
+
|
|
1246
|
+
return [word, i.reason ?? i.error ?? i.conflict ?? cause];
|
|
1247
|
+
}
|
|
1248
|
+
|
|
1249
|
+
/**
|
|
1250
|
+
* An item's line of run-log.csv (Harness-PRD §11): its result word (`paused` for the usage
|
|
1251
|
+
* limit, `no-result` for a session cut short), why, and what its sessions spent. A missing
|
|
1252
|
+
* value is an empty field.
|
|
1253
|
+
*/
|
|
1254
|
+
export const runRow = (mode) => (i) => [
|
|
1255
|
+
i.id,
|
|
1256
|
+
mode,
|
|
1257
|
+
...outcome(i),
|
|
1258
|
+
i.cost_usd,
|
|
1259
|
+
i.tokens?.input,
|
|
1260
|
+
i.tokens?.output,
|
|
1261
|
+
i.tokens?.cache_read,
|
|
1262
|
+
i.turns,
|
|
1263
|
+
i.duration_ms,
|
|
1264
|
+
i.handoffs ?? 0,
|
|
1265
|
+
i.process,
|
|
1266
|
+
];
|
|
1267
|
+
|
|
1268
|
+
/**
|
|
1269
|
+
* What rides on an item's line beside its result: its handoffs, and its broken steps (a
|
|
1270
|
+
* skipped step 13, a move with no green verify).
|
|
1271
|
+
* @param {any} r
|
|
1272
|
+
* @param {string | null} [note]
|
|
1273
|
+
*/
|
|
1274
|
+
const extras = (r, note = null) => {
|
|
1275
|
+
const notes = [r?.skipped && OUTSIDE_IMPLEMENT, note].filter(Boolean).join('; ');
|
|
1276
|
+
|
|
1277
|
+
return {
|
|
1278
|
+
...(r?.handoffs ? { handoffs: r.handoffs } : {}),
|
|
1279
|
+
...(notes ? { process: notes } : {}),
|
|
1280
|
+
...(r?.lessons ? { lessons: r.lessons } : {}),
|
|
1281
|
+
};
|
|
1282
|
+
};
|
|
1204
1283
|
|
|
1205
1284
|
/** Every claimed item in its worktree at once; blocks recorded on main, verified ones merged. */
|
|
1206
1285
|
async function runWorktrees(claims, ctx) {
|
|
@@ -1210,7 +1289,8 @@ async function runWorktrees(claims, ctx) {
|
|
|
1210
1289
|
const done = (r) => () =>
|
|
1211
1290
|
verified(r.dir, r.id) || existsSync(path.join(r.dir, '.gcf-session', 'outcome.json'));
|
|
1212
1291
|
const runs = await Promise.all(ready.map((r) => session({ ...r, done: done(r) }, ctx)));
|
|
1213
|
-
|
|
1292
|
+
// Refused before a session: the metrics keep them out of the share (§11).
|
|
1293
|
+
const items = failed.map((f) => ({ ...f, started: false }));
|
|
1214
1294
|
for (const r of runs) {
|
|
1215
1295
|
const outcome = await outcomeOf(r.dir);
|
|
1216
1296
|
if (outcome?.block) items.push(await recordBlock(r, outcome, core));
|
|
@@ -1287,6 +1367,7 @@ export async function run(
|
|
|
1287
1367
|
try {
|
|
1288
1368
|
const items =
|
|
1289
1369
|
runMode === 'main' ? await runMain(projectDir, ids, ctx) : await runWorktrees(claims, ctx);
|
|
1370
|
+
logMetrics(projectDir, 'run-log.csv', items.map(runRow(runMode)));
|
|
1290
1371
|
|
|
1291
1372
|
return { orphans, mode: runMode, items };
|
|
1292
1373
|
} finally {
|
package/extension.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://raw.githubusercontent.com/lee-to/ai-factory/2.x/schemas/extension.schema.json",
|
|
3
3
|
"name": "gcf-harness-aif-ext",
|
|
4
|
-
"version": "0.9.
|
|
4
|
+
"version": "0.9.4",
|
|
5
5
|
"description": "GCF orchestration harness for Claude Code: one-command card pipeline, readset guard",
|
|
6
6
|
"skills": [
|
|
7
7
|
"skills/gcf-run",
|
package/hooks/implement-guard.js
CHANGED
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
// shortcut: a write through Bash is not seen here, nor by the run's check: the diff review is.
|
|
11
11
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
12
12
|
import path from 'node:path';
|
|
13
|
+
import { logMetrics } from './metrics.js';
|
|
13
14
|
|
|
14
15
|
const WRITES = ['Edit', 'Write', 'MultiEdit', 'NotebookEdit'];
|
|
15
16
|
const PLANNING_DIRS = ['gcf', '.ai-factory', '.gcf-session'];
|
|
@@ -70,17 +71,43 @@ const ENDS = new RegExp(String.raw`(^|[;&|(])\s*${CLI}gcf\s+(block|submit|close)
|
|
|
70
71
|
const endsRun = (input) =>
|
|
71
72
|
['Bash', 'PowerShell'].includes(input.tool_name) &&
|
|
72
73
|
ENDS.test(String(input.tool_input?.command ?? ''));
|
|
74
|
+
// A move past the DoD gate (Methodology §6) is the human's: a session that ran /gcf-run never
|
|
75
|
+
// makes one, even after its end move (a close the core refused must not free it for a skip).
|
|
76
|
+
// The flag may stand anywhere after the move, before the id too, and a line may go on past a
|
|
77
|
+
// `\` or a PowerShell backtick.
|
|
78
|
+
// shortcut: the call forms are CLI's (ai-factory, npx, a path, .cmd); an env prefix, `npx -y`,
|
|
79
|
+
// `bash -c` or a quoted path slip past, as they do past ENDS; widen CLI if a run shows one.
|
|
80
|
+
const SKIPS = new RegExp(
|
|
81
|
+
String.raw`(^|[;&|(])\s*${CLI}gcf\s+(submit|close)\b[^\n]*?[\s'"]--unverified\b`,
|
|
82
|
+
'm',
|
|
83
|
+
);
|
|
84
|
+
export const skipsVerify = (input) =>
|
|
85
|
+
['Bash', 'PowerShell'].includes(input.tool_name) &&
|
|
86
|
+
SKIPS.test(String(input.tool_input?.command ?? '').replace(/[\\`]\r?\n/g, ' '));
|
|
87
|
+
const NO_SKIP =
|
|
88
|
+
"a move without a green gcf verify (--unverified) is the human's call: run ai-factory gcf " +
|
|
89
|
+
'verify <id> until it is green, or block the item with what keeps the DoD red';
|
|
73
90
|
|
|
74
91
|
/**
|
|
75
92
|
* The session's state after `input`, and the verdict on it: `{state, allow, reason?}`. `state`
|
|
76
93
|
* is `{run, implement}`; /gcf-run and aif-implement set it, the move that ends the item clears
|
|
77
94
|
* `run`, so the operator's own edits after a blocked run are free again. `implement` stays: the
|
|
78
95
|
* run's check reads it for a session resumed after a limit, whose stream has no Skill call.
|
|
96
|
+
* `gated` stays after the end move of a run: a move past the verify gate is refused in such a
|
|
97
|
+
* session, and its state is kept.
|
|
79
98
|
*/
|
|
80
|
-
export
|
|
99
|
+
export const decide = (input, projectDir, state) =>
|
|
100
|
+
input && (state?.run || state?.gated) && skipsVerify(input)
|
|
101
|
+
? { state: { ...state }, allow: false, reason: NO_SKIP }
|
|
102
|
+
: decideMove(input, projectDir, state);
|
|
103
|
+
|
|
104
|
+
/** `decide` for any input but a move past the verify gate. */
|
|
105
|
+
function decideMove(input, projectDir, state) {
|
|
106
|
+
const old = state ?? {};
|
|
107
|
+
// The end move: the run is over, its gate on a skip of the verify stays.
|
|
81
108
|
if (input && endsRun(input))
|
|
82
|
-
return { state:
|
|
83
|
-
const next =
|
|
109
|
+
return { state: stateOf(false, old.implement, old.run || old.gated), allow: true };
|
|
110
|
+
const next = stateOf(old.run, old.implement, old.gated);
|
|
84
111
|
if (!input) return { state: next, allow: true };
|
|
85
112
|
next.run ||= started(input, 'gcf-run');
|
|
86
113
|
next.implement ||= started(input, 'aif-implement');
|
|
@@ -89,6 +116,13 @@ export function decide(input, projectDir, state) {
|
|
|
89
116
|
return { state: next, ...(guarded ? writeVerdict(input, projectDir) : { allow: true }) };
|
|
90
117
|
}
|
|
91
118
|
|
|
119
|
+
/** A session's state; `gated` only when set, so a session that never ran /gcf-run has none. */
|
|
120
|
+
const stateOf = (run, implement, gated) => ({
|
|
121
|
+
run: Boolean(run),
|
|
122
|
+
implement: Boolean(implement),
|
|
123
|
+
...(gated ? { gated: true } : {}),
|
|
124
|
+
});
|
|
125
|
+
|
|
92
126
|
/** A write before aif-implement: code is denied, and so is a tick in markdown. */
|
|
93
127
|
function writeVerdict(input, projectDir) {
|
|
94
128
|
const file = input.tool_input?.file_path ?? input.tool_input?.notebook_path;
|
|
@@ -107,7 +141,7 @@ export const stateFile = (projectDir, sessionId) =>
|
|
|
107
141
|
? path.join(projectDir, '.gcf-session', 'guard', `${sessionId}.json`)
|
|
108
142
|
: null;
|
|
109
143
|
|
|
110
|
-
/** A session's state: `{run, implement}`, or null without one. */
|
|
144
|
+
/** A session's state: `{run, implement, gated?}`, or null without one. */
|
|
111
145
|
export function readState(projectDir, sessionId) {
|
|
112
146
|
const file = stateFile(projectDir, sessionId);
|
|
113
147
|
try {
|
|
@@ -129,7 +163,8 @@ if (import.meta.url === `file://${entry}` || entry?.endsWith('implement-guard.js
|
|
|
129
163
|
const file = stateFile(projectDir, input?.session_id);
|
|
130
164
|
const changed =
|
|
131
165
|
verdict.state.run !== Boolean(old?.run) ||
|
|
132
|
-
verdict.state.implement !== Boolean(old?.implement)
|
|
166
|
+
verdict.state.implement !== Boolean(old?.implement) ||
|
|
167
|
+
Boolean(verdict.state.gated) !== Boolean(old?.gated);
|
|
133
168
|
if (file && changed) {
|
|
134
169
|
mkdirSync(path.dirname(file), { recursive: true });
|
|
135
170
|
writeFileSync(file, `${JSON.stringify(verdict.state)}\n`);
|
|
@@ -137,6 +172,10 @@ if (import.meta.url === `file://${entry}` || entry?.endsWith('implement-guard.js
|
|
|
137
172
|
|
|
138
173
|
// A prompt is never refused: exit 2 there would drop the operator's message.
|
|
139
174
|
if (!verdict.allow && input?.hook_event_name !== 'UserPromptSubmit') {
|
|
175
|
+
const item = process.env.GCF_RUN_ITEM ?? '';
|
|
176
|
+
// A skip of the verify gate counts apart from a write before aif-implement.
|
|
177
|
+
const hook = skipsVerify(input) ? 'verify' : 'implement';
|
|
178
|
+
logMetrics(projectDir, 'guard-log.csv', [[item, hook, input?.tool_name]]);
|
|
140
179
|
console.error(`gcf implement-guard: ${verdict.reason}`);
|
|
141
180
|
process.exit(2);
|
|
142
181
|
}
|
package/hooks/metrics.js
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
// The harness's lines in gcf/.metrics (Harness-PRD §11), in the main checkout's logs: a
|
|
2
|
+
// worktree's own gcf/.metrics is thrown away when its branch merges. The same layout as the
|
|
3
|
+
// core's lib/metrics.js; the harness calls no core module, only its CLI.
|
|
4
|
+
import { spawnSync } from 'node:child_process';
|
|
5
|
+
import { appendFileSync, existsSync, mkdirSync } from 'node:fs';
|
|
6
|
+
import path from 'node:path';
|
|
7
|
+
|
|
8
|
+
/** gcf/.metrics of the main checkout of `projectDir`; its own outside a repository. */
|
|
9
|
+
export function metricsDir(projectDir) {
|
|
10
|
+
const res = spawnSync('git', ['rev-parse', '--git-common-dir'], {
|
|
11
|
+
cwd: projectDir,
|
|
12
|
+
encoding: 'utf8',
|
|
13
|
+
});
|
|
14
|
+
const common = res.status === 0 ? path.resolve(projectDir, res.stdout.trim()) : '';
|
|
15
|
+
const main = path.basename(common) === '.git' ? path.dirname(common) : projectDir;
|
|
16
|
+
|
|
17
|
+
return path.join(main, 'gcf', '.metrics');
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
/** A field as csv takes it: no comma, no line break, at most 120 characters. */
|
|
21
|
+
const field = (v) =>
|
|
22
|
+
String(v ?? '')
|
|
23
|
+
.replace(/[,\r\n]+/g, ';')
|
|
24
|
+
.slice(0, 120);
|
|
25
|
+
|
|
26
|
+
/**
|
|
27
|
+
* Append timestamped lines to a log; a project without gcf/ keeps none. Best effort: a log
|
|
28
|
+
* that cannot be written never stops a run or a hook.
|
|
29
|
+
*/
|
|
30
|
+
export function logMetrics(projectDir, file, rows) {
|
|
31
|
+
try {
|
|
32
|
+
const dir = metricsDir(projectDir);
|
|
33
|
+
if (!rows.length || !existsSync(path.dirname(dir))) return;
|
|
34
|
+
mkdirSync(dir, { recursive: true });
|
|
35
|
+
const at = new Date().toISOString();
|
|
36
|
+
const text = rows.map((r) => `${[at, ...r].map(field).join(',')}\n`).join('');
|
|
37
|
+
appendFileSync(path.join(dir, file), text, 'utf8');
|
|
38
|
+
} catch {
|
|
39
|
+
// No log this time.
|
|
40
|
+
}
|
|
41
|
+
}
|
package/hooks/readset-guard.js
CHANGED
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
// tool for a solo developer, not a security perimeter.
|
|
9
9
|
import { readFileSync, readdirSync, statSync, utimesSync } from 'node:fs';
|
|
10
10
|
import path from 'node:path';
|
|
11
|
+
import { logMetrics } from './metrics.js';
|
|
11
12
|
|
|
12
13
|
function readStdin() {
|
|
13
14
|
try {
|
|
@@ -145,8 +146,12 @@ if (import.meta.url === `file://${entry}` || entry?.endsWith('readset-guard.js')
|
|
|
145
146
|
// not found there would allow every read.
|
|
146
147
|
const projectDir = process.env.CLAUDE_PROJECT_DIR ?? input?.cwd ?? process.cwd();
|
|
147
148
|
keepSessionsAlive(projectDir);
|
|
148
|
-
const
|
|
149
|
+
const manifest = readManifest(projectDir);
|
|
150
|
+
const verdict = decide(input, projectDir, manifest);
|
|
149
151
|
if (!verdict.allow) {
|
|
152
|
+
// Harness-PRD §11, metric 3: each firing is a hole in the readset or a wandering agent.
|
|
153
|
+
const item = process.env.GCF_RUN_ITEM ?? manifest?.item ?? '';
|
|
154
|
+
logMetrics(projectDir, 'guard-log.csv', [[item, 'readset', input?.tool_name]]);
|
|
150
155
|
console.error(`gcf readset-guard: ${verdict.reason}`);
|
|
151
156
|
process.exit(2);
|
|
152
157
|
}
|
package/package.json
CHANGED
package/skills/gcf-run/SKILL.md
CHANGED
|
@@ -1,80 +1,80 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: gcf-run
|
|
3
|
-
description: Drive one or more GCF work items through plan, implement and DoD in a single run - gates before tokens, a converging plan, readset discipline, then submit, close or block. Use when the user says "прогони задачу" / "прогони карточку" / "run the item".
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
mode: pipeline
|
|
7
|
-
|
|
8
|
-
purpose:
|
|
9
|
-
- drive a work item through gate, readset, plan, implement and its DoD with the stop rules checked before each paid stage
|
|
10
|
-
- end every item in a state the core records: `submit` (waits for the human), `close` (gate auto, done) or `block` (needs the human)
|
|
11
|
-
- outside `--worktree`, mark each item running after its gate passes: `ai-factory gcf claim <id>`; a move clears the mark, and an item left without one gets `ai-factory gcf release <id>`
|
|
12
|
-
- a claimed item takes only its session's moves: run `submit`, `close` and `block` on it with `--claimed`
|
|
13
|
-
|
|
14
|
-
inputs:
|
|
15
|
-
- 1 or more work item or design unit ids; a batch shares one build and one playtest list
|
|
16
|
-
- `--worktree` when `ai-factory gcf-harness run` started this session inside the item's worktree
|
|
17
|
-
- `--handoff` when the harness started this session after the last one filled its context: the plan's `## Handoff` says where it stopped
|
|
18
|
-
- `--no-lessons` when the harness asks this session's lessons itself, in a turn after it ends
|
|
19
|
-
|
|
20
|
-
preconditions:
|
|
21
|
-
- `gcf/` exists and `ai-factory gcf audit` runs (the gcf-aif-ext extension provides it)
|
|
22
|
-
|
|
23
|
-
forbidden_behaviors:
|
|
24
|
-
- do not give a verdict or a decision: `verdict` and `decide` belong to the human, this pipeline stops at `submit`
|
|
25
|
-
- do not read a `gcf/` file outside the readset from `ai-factory gcf context`
|
|
26
|
-
- do not continue past a failed gate: report the one-line reason, delete `.gcf-session/readset.json`, and stop - a stop is a normal outcome
|
|
27
|
-
- do not compare the audit numbers yourself: the gate command is the only judge of whether an item may go
|
|
28
|
-
- do not ask the human about a reversible choice during planning: pick the safe default and write it into the plan
|
|
29
|
-
- answer aif-plan's own questions (tests, logging, docs, other preferences) yourself from the item, its DoD and the project's rules, and write each answer into the plan
|
|
30
|
-
- a plan pass sorts each finding as blocking or polish, and reports how many of each it found
|
|
31
|
-
- blocking: a wrong or missing behavior, a missing or wrong task, a broken pillar invariant, a conflict with a project rule, a task no check can prove
|
|
32
|
-
- polish: wording, splitting or merging tasks, their order, more detail - the pass applies it, and it does not keep the plan from converging
|
|
33
|
-
- do not edit `status:` or `gcf/LIVE.md` by hand: `submit`, `close` and `block` own them
|
|
34
|
-
- do not write a plan task's first implementation outside the aif-implement skill: it ticks the task done in the plan, and the board shows the item's progress from those ticks
|
|
35
|
-
- a fix for a red DoD or a review finding goes in directly, without aif-implement
|
|
36
|
-
- do not let aif-implement commit, stash or merge: answer its questions yourself
|
|
37
|
-
- at aif-implement's uncommitted-changes question pick none of its three options - not "No", which stashes - and go on with the plan
|
|
38
|
-
- at aif-implement's commit checkpoints go on to the next task; skip its rules and docs checkpoints; keep the plan file; at its merge question answer that you handle it
|
|
39
|
-
- a ticked task whose code aif-implement's re-entry check finds missing stays pending and gets implemented, with no question to the human
|
|
40
|
-
- at aif-implement's last question, verify or commit, pick neither: go on with step 13, which verifies, and steps 14-16, which review and commit
|
|
41
|
-
- do not leave a task ticked that the tune note or the answer reopens: untick it and add the note before step 13, or aif-implement's re-entry check ticks it again over the old code
|
|
42
|
-
- do not route a command the permission system refused through a script, a wrapper or another spelling or shell (a full exe path, `cmd /c`, `node`, `npm run`)
|
|
43
|
-
- do not write a refused command into a DoD command or a config that `ai-factory gcf` runs
|
|
44
|
-
- a dedicated tool doing the same job (Read, Grep, Glob, Edit) is no wrapper, nor is a refused pipe or `;` chain split into single allowed commands
|
|
45
|
-
- a refusal none of those replaces is a stop: run `ai-factory gcf block <id> --reason "needs permission: <command>"` and stop the item; under `--worktree` step 22 records it instead
|
|
46
|
-
- a call the handoff guard denies with "context at" is no refusal: block nothing, write the `## Handoff` it asks for and end the reply
|
|
47
|
-
- a read the readset guard denies is no refusal either: drop that read, and widen the readset only through the item's `system:` or `depends:`
|
|
48
|
-
- delete one file with `rm -f <file>` or `Remove-Item -Force <file>`, never with `-r`: the deny rules refuse a recursive delete
|
|
49
|
-
|
|
50
|
-
workflow:
|
|
51
|
-
1. session ritual: delete a `.gcf-session/readset.json` left by a killed run; run `ai-factory gcf integrate` to merge the verified branches a dead run left; run `ai-factory gcf audit --json`; read `gcf/INDEX.md`; report the audit first line
|
|
52
|
-
2. for a unit still in `gcf/design/`: take it with the gcf-card skill first
|
|
53
|
-
3. for a feature id: replace it with its `ready` children from the audit report's `items` (`parent` names the feature) that are not in `open_depends`; a feature in `awaiting-decision` starts none of them
|
|
54
|
-
4. of those children, keep one per subsystem unless every child sharing it says `parallel: ok`; the rest wait for the next run; an empty list: report "nothing to run" and stop
|
|
55
|
-
5. gate: run `ai-factory gcf-harness gate <item-ids...>` over the whole batch; a non-zero exit is a failed gate, its stderr line is the reason
|
|
56
|
-
6. readset: run `ai-factory gcf context <item-id> --json`; write `{"item":"<id>","files": ctx.files.map(f => f.path)}` to `.gcf-session/readset.json` - paths, not objects; `over_budget: true` stops the run with a D1 question naming the subsystem
|
|
57
|
-
7. plan: run aif-plan full for the item into its `plan:` path, by default `.ai-factory/plans/<id>.md`, reading only the readset files; an existing plan (the item came back after a tune, an unblock or a continue) is kept, not rewritten
|
|
58
|
-
8. improve the plan in passes, each pass a fresh subagent that reads only the plan and the readset and applies aif-improve without questions, writing each default it picks into the plan
|
|
59
|
-
9. each pass also checks every task against the `invariant` of each pillar in `gcf/PILLARS.md`: the pass reworks a task that breaks one, and a conflict no rework removes is a pillar conflict of step 12
|
|
60
|
-
10. after each pass run `ai-factory gcf plan check <id>`; a failure blocks the item with its reason; a pass with no blocking finding ends the loop - the plan converged, its polish applied
|
|
61
|
-
11. passes <= `plan_passes` from `gcf/config.yml` (3 by default); a blocking finding left after the last pass: run `ai-factory gcf block <id> --reason "plan did not converge: <the finding>"` and stop
|
|
62
|
-
12. a question only the human can answer - a save format, a public API, a new dependency, a pillar conflict: run `ai-factory gcf block <id> --reason "<question>"` and stop
|
|
63
|
-
13. run the aif-implement skill on `@<plan path>`, write the item's `live:` delta, then run `ai-factory gcf verify <id>` and check the item's negative constraints; a non-zero exit is a red DoD; attempts <= `dod_attempts`, each with a diagnosis
|
|
64
|
-
14. review the diff with a fresh subagent; fix what it finds before the commit; DoD still red after the attempts: commit no code, run `ai-factory gcf block <id> --reason "<diagnosis>"` - its commit carries the item file with its delta
|
|
65
|
-
15. the reviewer also answers the `review_question` of each pillar in `gcf/PILLARS.md` over the diff: an answer that breaks the pillar is a finding
|
|
66
|
-
16. commit the implementation, its plan and the `live:` delta via aif-commit
|
|
67
|
-
17. gate `verdict` or `decision`: run `ai-factory gcf submit <id>`; gate `auto`: run `ai-factory gcf close <id>` - a soft unit closes unplayed
|
|
68
|
-
18. a dependency still waiting for its verdict refuses the close: run `ai-factory gcf block <id> --reason "waits for <dep>"`
|
|
69
|
-
19. close the run: delete `.gcf-session/readset.json`, run `ai-factory gcf playtest session` and hand its checklist to the human, with one report row per plan pass: pass, blocking, polish, result
|
|
70
|
-
20. after a tune verdict or an unblock, re-enter this pipeline at step 5: the gates re-run because the counters moved
|
|
71
|
-
21. with `--worktree`: skip the integrate of step 1, steps 2-4 and the playtest of step 19, and change no `gcf/` file but the item's `live:`
|
|
72
|
-
22. with `--worktree`, every stop - a failed gate, an over budget readset, a failed plan check, a refused command, steps 11, 12, 14, 18 - writes `{"block": "<reason>"}` to `.gcf-session/outcome.json` and ends the session
|
|
73
|
-
23. with `--worktree`, run no `ai-factory gcf block`, `submit` or `close`: the core refuses them on a `work/` branch, and the harness records the move on main
|
|
74
|
-
24. with `--worktree`, step 17 is: commit everything but `gcf/.metrics`, then run `ai-factory gcf verify <id> --mark`; the harness merges the branch
|
|
75
|
-
25. with `--handoff`: run steps 1, 5 and 6, read the plan's `## Handoff`, skip steps 7-11 unless it says the plan passes were not done, then go on at step 13; a plan with no `## Handoff` goes on at step 13 from its ticks
|
|
76
|
-
26. with `--handoff`, take the review findings and the DoD diagnosis of `## Handoff` into steps 13-15, and delete the section before the commit of step 16
|
|
77
|
-
27. without `--no-lessons` and without `--worktree`, end the run with the lessons: run `ai-factory gcf lessons prompt <id>` for each item of the run and do what it says
|
|
78
|
-
|
|
79
|
-
invocation:
|
|
80
|
-
- run when the user names a work item or a unit to drive; batches share steps 1 and 19
|
|
1
|
+
---
|
|
2
|
+
name: gcf-run
|
|
3
|
+
description: Drive one or more GCF work items through plan, implement and DoD in a single run - gates before tokens, a converging plan, readset discipline, then submit, close or block. Use when the user says "прогони задачу" / "прогони карточку" / "run the item".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
mode: pipeline
|
|
7
|
+
|
|
8
|
+
purpose:
|
|
9
|
+
- drive a work item through gate, readset, plan, implement and its DoD with the stop rules checked before each paid stage
|
|
10
|
+
- end every item in a state the core records: `submit` (waits for the human), `close` (gate auto, done) or `block` (needs the human)
|
|
11
|
+
- outside `--worktree`, mark each item running after its gate passes: `ai-factory gcf claim <id>`; a move clears the mark, and an item left without one gets `ai-factory gcf release <id>`
|
|
12
|
+
- a claimed item takes only its session's moves: run `submit`, `close` and `block` on it with `--claimed`
|
|
13
|
+
|
|
14
|
+
inputs:
|
|
15
|
+
- 1 or more work item or design unit ids; a batch shares one build and one playtest list
|
|
16
|
+
- `--worktree` when `ai-factory gcf-harness run` started this session inside the item's worktree
|
|
17
|
+
- `--handoff` when the harness started this session after the last one filled its context: the plan's `## Handoff` says where it stopped
|
|
18
|
+
- `--no-lessons` when the harness asks this session's lessons itself, in a turn after it ends
|
|
19
|
+
|
|
20
|
+
preconditions:
|
|
21
|
+
- `gcf/` exists and `ai-factory gcf audit` runs (the gcf-aif-ext extension provides it)
|
|
22
|
+
|
|
23
|
+
forbidden_behaviors:
|
|
24
|
+
- do not give a verdict or a decision: `verdict` and `decide` belong to the human, this pipeline stops at `submit`
|
|
25
|
+
- do not read a `gcf/` file outside the readset from `ai-factory gcf context`
|
|
26
|
+
- do not continue past a failed gate: report the one-line reason, delete `.gcf-session/readset.json`, and stop - a stop is a normal outcome
|
|
27
|
+
- do not compare the audit numbers yourself: the gate command is the only judge of whether an item may go
|
|
28
|
+
- do not ask the human about a reversible choice during planning: pick the safe default and write it into the plan
|
|
29
|
+
- answer aif-plan's own questions (tests, logging, docs, other preferences) yourself from the item, its DoD and the project's rules, and write each answer into the plan
|
|
30
|
+
- a plan pass sorts each finding as blocking or polish, and reports how many of each it found
|
|
31
|
+
- blocking: a wrong or missing behavior, a missing or wrong task, a broken pillar invariant, a conflict with a project rule, a task no check can prove
|
|
32
|
+
- polish: wording, splitting or merging tasks, their order, more detail - the pass applies it, and it does not keep the plan from converging
|
|
33
|
+
- do not edit `status:` or `gcf/LIVE.md` by hand: `submit`, `close` and `block` own them
|
|
34
|
+
- do not write a plan task's first implementation outside the aif-implement skill: it ticks the task done in the plan, and the board shows the item's progress from those ticks
|
|
35
|
+
- a fix for a red DoD or a review finding goes in directly, without aif-implement
|
|
36
|
+
- do not let aif-implement commit, stash or merge: answer its questions yourself
|
|
37
|
+
- at aif-implement's uncommitted-changes question pick none of its three options - not "No", which stashes - and go on with the plan
|
|
38
|
+
- at aif-implement's commit checkpoints go on to the next task; skip its rules and docs checkpoints; keep the plan file; at its merge question answer that you handle it
|
|
39
|
+
- a ticked task whose code aif-implement's re-entry check finds missing stays pending and gets implemented, with no question to the human
|
|
40
|
+
- at aif-implement's last question, verify or commit, pick neither: go on with step 13, which verifies, and steps 14-16, which review and commit
|
|
41
|
+
- do not leave a task ticked that the tune note or the answer reopens: untick it and add the note before step 13, or aif-implement's re-entry check ticks it again over the old code
|
|
42
|
+
- do not route a command the permission system refused through a script, a wrapper or another spelling or shell (a full exe path, `cmd /c`, `node`, `npm run`)
|
|
43
|
+
- do not write a refused command into a DoD command or a config that `ai-factory gcf` runs
|
|
44
|
+
- a dedicated tool doing the same job (Read, Grep, Glob, Edit) is no wrapper, nor is a refused pipe or `;` chain split into single allowed commands
|
|
45
|
+
- a refusal none of those replaces is a stop: run `ai-factory gcf block <id> --reason "needs permission: <command>"` and stop the item; under `--worktree` step 22 records it instead
|
|
46
|
+
- a call the handoff guard denies with "context at" is no refusal: block nothing, write the `## Handoff` it asks for and end the reply
|
|
47
|
+
- a read the readset guard denies is no refusal either: drop that read, and widen the readset only through the item's `system:` or `depends:`
|
|
48
|
+
- delete one file with `rm -f <file>` or `Remove-Item -Force <file>`, never with `-r`: the deny rules refuse a recursive delete
|
|
49
|
+
|
|
50
|
+
workflow:
|
|
51
|
+
1. session ritual: delete a `.gcf-session/readset.json` left by a killed run; run `ai-factory gcf integrate` to merge the verified branches a dead run left; run `ai-factory gcf audit --json`; read `gcf/INDEX.md`; report the audit first line
|
|
52
|
+
2. for a unit still in `gcf/design/`: take it with the gcf-card skill first
|
|
53
|
+
3. for a feature id: replace it with its `ready` children from the audit report's `items` (`parent` names the feature) that are not in `open_depends`; a feature in `awaiting-decision` starts none of them
|
|
54
|
+
4. of those children, keep one per subsystem unless every child sharing it says `parallel: ok`; the rest wait for the next run; an empty list: report "nothing to run" and stop
|
|
55
|
+
5. gate: run `ai-factory gcf-harness gate <item-ids...>` over the whole batch; a non-zero exit is a failed gate, its stderr line is the reason
|
|
56
|
+
6. readset: run `ai-factory gcf context <item-id> --json`; write `{"item":"<id>","files": ctx.files.map(f => f.path)}` to `.gcf-session/readset.json` - paths, not objects; `over_budget: true` stops the run with a D1 question naming the subsystem
|
|
57
|
+
7. plan: run aif-plan full for the item into its `plan:` path, by default `.ai-factory/plans/<id>.md`, reading only the readset files; an existing plan (the item came back after a tune, an unblock or a continue) is kept, not rewritten
|
|
58
|
+
8. improve the plan in passes, each pass a fresh subagent that reads only the plan and the readset and applies aif-improve without questions, writing each default it picks into the plan
|
|
59
|
+
9. each pass also checks every task against the `invariant` of each pillar in `gcf/PILLARS.md`: the pass reworks a task that breaks one, and a conflict no rework removes is a pillar conflict of step 12
|
|
60
|
+
10. after each pass run `ai-factory gcf plan check <id>`; a failure blocks the item with its reason; a pass with no blocking finding ends the loop - the plan converged, its polish applied
|
|
61
|
+
11. passes <= `plan_passes` from `gcf/config.yml` (3 by default); a blocking finding left after the last pass: run `ai-factory gcf block <id> --reason "plan did not converge: <the finding>"` and stop
|
|
62
|
+
12. a question only the human can answer - a save format, a public API, a new dependency, a pillar conflict: run `ai-factory gcf block <id> --reason "<question>"` and stop
|
|
63
|
+
13. run the aif-implement skill on `@<plan path>`, write the item's `live:` delta, then run `ai-factory gcf verify <id>` and check the item's negative constraints; a non-zero exit is a red DoD; attempts <= `dod_attempts`, each with a diagnosis
|
|
64
|
+
14. review the diff with a fresh subagent; fix what it finds before the commit; DoD still red after the attempts: commit no code, run `ai-factory gcf block <id> --reason "<diagnosis>"` - its commit carries the item file with its delta
|
|
65
|
+
15. the reviewer also answers the `review_question` of each pillar in `gcf/PILLARS.md` over the diff: an answer that breaks the pillar is a finding
|
|
66
|
+
16. after a review fix, run `ai-factory gcf verify <id>` again until green - `submit` and `close` refuse changed code; then commit the implementation, its plan and the `live:` delta via aif-commit
|
|
67
|
+
17. gate `verdict` or `decision`: run `ai-factory gcf submit <id>`; gate `auto`: run `ai-factory gcf close <id>` - a soft unit closes unplayed; never pass `--unverified`: a move without a green verify is the human's call
|
|
68
|
+
18. a dependency still waiting for its verdict refuses the close: run `ai-factory gcf block <id> --reason "waits for <dep>"`
|
|
69
|
+
19. close the run: delete `.gcf-session/readset.json`, run `ai-factory gcf playtest session` and hand its checklist to the human, with one report row per plan pass: pass, blocking, polish, result
|
|
70
|
+
20. after a tune verdict or an unblock, re-enter this pipeline at step 5: the gates re-run because the counters moved
|
|
71
|
+
21. with `--worktree`: skip the integrate of step 1, steps 2-4 and the playtest of step 19, and change no `gcf/` file but the item's `live:`
|
|
72
|
+
22. with `--worktree`, every stop - a failed gate, an over budget readset, a failed plan check, a refused command, steps 11, 12, 14, 18 - writes `{"block": "<reason>"}` to `.gcf-session/outcome.json` and ends the session
|
|
73
|
+
23. with `--worktree`, run no `ai-factory gcf block`, `submit` or `close`: the core refuses them on a `work/` branch, and the harness records the move on main
|
|
74
|
+
24. with `--worktree`, step 17 is: commit everything but `gcf/.metrics`, then run `ai-factory gcf verify <id> --mark`; the harness merges the branch
|
|
75
|
+
25. with `--handoff`: run steps 1, 5 and 6, read the plan's `## Handoff`, skip steps 7-11 unless it says the plan passes were not done, then go on at step 13; a plan with no `## Handoff` goes on at step 13 from its ticks
|
|
76
|
+
26. with `--handoff`, take the review findings and the DoD diagnosis of `## Handoff` into steps 13-15, and delete the section before the commit of step 16
|
|
77
|
+
27. without `--no-lessons` and without `--worktree`, end the run with the lessons: run `ai-factory gcf lessons prompt <id>` for each item of the run and do what it says
|
|
78
|
+
|
|
79
|
+
invocation:
|
|
80
|
+
- run when the user names a work item or a unit to drive; batches share steps 1 and 19
|