@sassywares/skills 0.5.16 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONCEPTS.md +1 -1
- package/README.md +3 -4
- package/package.json +1 -1
- package/skills/sassy-babysit/SKILL.md +2 -0
- package/skills/sassy-commit/SKILL.md +2 -0
- package/skills/sassy-commit-push-pr/SKILL.md +2 -0
- package/skills/sassy-cycle/SKILL.md +2 -0
- package/skills/sassy-debug/SKILL.md +2 -0
- package/skills/sassy-handbook/SKILL.md +2 -0
- package/skills/sassy-handoff/SKILL.md +50 -24
- package/skills/sassy-lfg/SKILL.md +3 -1
- package/skills/sassy-plan/SKILL.md +2 -0
- package/skills/sassy-review/SKILL.md +2 -0
- package/skills/sassy-scout/SKILL.md +3 -1
- package/skills/sassy-setup/SKILL.md +2 -0
- package/skills/sassy-setup/assets/sassywares-readme.md +1 -1
- package/skills/sassy-setup/references/harness.md +33 -0
- package/skills/sassy-ship/SKILL.md +2 -0
- package/skills/sassy-skill/SKILL.md +15 -10
- package/skills/sassy-skill/references/create.md +4 -4
- package/skills/sassy-skill/references/retune.md +2 -2
- package/skills/sassy-skill/references/verify.md +2 -2
- package/skills/sassy-validate/SKILL.md +2 -0
- package/skills/sassy-work/SKILL.md +4 -8
- package/skills/sassy-compare/SKILL.md +0 -95
- package/skills/sassy-compare/scripts/compare.py +0 -493
package/CONCEPTS.md
CHANGED
|
@@ -58,7 +58,7 @@ A settled in-context brief produced by `sassy-plan` when the user picks "start w
|
|
|
58
58
|
|
|
59
59
|
## Handoff
|
|
60
60
|
|
|
61
|
-
`sassy-handoff`
|
|
61
|
+
`sassy-handoff` writes a self-contained handoff file under `/tmp/sassy-handoff-<uid>/<harness>/` and gives the user its path and a paste-ready prompt. A skill cannot start a session, so the user opens the fresh one; the file is the only bridge, in any harness.
|
|
62
62
|
|
|
63
63
|
## Sync
|
|
64
64
|
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
**Lean and mean skills for shipping fast — with full control and 100% reliability.**
|
|
4
4
|
|
|
5
|
-
Sassy Skills is a suite of explicit-invocation Agent Skills. It follows the Agent Skills format, so it works with any harness that can load `SKILL.md` workflows.
|
|
5
|
+
Sassy Skills is a suite of explicit-invocation Agent Skills. It follows the Agent Skills format, so it works with any harness that can load `SKILL.md` workflows. Where harnesses differ — how one skill loads another, how agents are dispatched, which command the user types — the skills follow one shared reference, `sassy-setup`'s `references/harness.md`, instead of assuming a harness. The subagents they dispatch come from [sassy-agents](https://github.com/sassywares/sassy-agents), one folder per harness.
|
|
6
6
|
|
|
7
7
|
It covers the whole path from a rough idea to a shipped PR, and from a bug report to a verified fix — without giving up human control at the moments that matter.
|
|
8
8
|
|
|
@@ -85,7 +85,7 @@ Debugging:
|
|
|
85
85
|
| **`sassy-lfg`** | Executes every phase of a plan, or one in-context brief, then closes out via `sassy-ship` after code approval. |
|
|
86
86
|
| **`sassy-cycle`** | Runs one unit of work through implement → review → validate → fix → re-check. |
|
|
87
87
|
| **`sassy-debug`** | Diagnoses a bug, gates the fix, fixes via `sassy-cycle`, then ships after code approval. Debug takes no mode and always leaves ship's review gate in place unless you invoke ship yourself with `mode:auto`. |
|
|
88
|
-
| **`sassy-handoff`** |
|
|
88
|
+
| **`sassy-handoff`** | Writes a self-contained handoff file to a temp directory and gives you its path plus a one-line prompt to paste into a fresh session, in any harness. |
|
|
89
89
|
|
|
90
90
|
### Code quality
|
|
91
91
|
|
|
@@ -115,8 +115,7 @@ Debugging:
|
|
|
115
115
|
| Skill | What it does |
|
|
116
116
|
|---|---|
|
|
117
117
|
| **`sassy-skill`** | Creates, retunes, or restructures Agent Skills. |
|
|
118
|
-
| **`sassy-setup`** | Detects the project topology and writes `.sassywares/config.yaml`; installs the snapshot tooling for a multi-repo workspace; commits the files it wrote. `mode:check` verifies an existing setup, including the vendored pin. |
|
|
119
|
-
| **`sassy-compare`** | Runs one prompt against two models in isolated worktrees, with optional per-side thinking levels, and compares output, code changes, timing, tokens, and errors. |
|
|
118
|
+
| **`sassy-setup`** | Detects the project topology and writes `.sassywares/config.yaml`; installs the sassy-agents subagents for your harness when missing; installs the snapshot tooling for a multi-repo workspace; commits the files it wrote. `mode:check` verifies an existing setup, including the vendored pin. |
|
|
120
119
|
|
|
121
120
|
## Project config
|
|
122
121
|
|
package/package.json
CHANGED
|
@@ -27,6 +27,8 @@ An explicit token always beats the config.
|
|
|
27
27
|
|
|
28
28
|
**One PR per run.** This skill watches the PR resolved from the argument or the current branch. A workspace fan-out is N runs, one per PR, and `pr.merge` never merges a fan-out — that needs an explicit say-so per PR.
|
|
29
29
|
|
|
30
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
31
|
+
|
|
30
32
|
## Step 1 — Resolve and arm
|
|
31
33
|
|
|
32
34
|
1. `gh repo view` must succeed, else say GitHub-only and stop.
|
|
@@ -12,6 +12,8 @@ disable-model-invocation: true
|
|
|
12
12
|
|
|
13
13
|
**Outcome:** Each logical change committed with an explicit file list and a message stating the outcome, and `git status` clean of those changes.
|
|
14
14
|
|
|
15
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
16
|
+
|
|
15
17
|
## Context
|
|
16
18
|
|
|
17
19
|
Gather this first, each command as its own shell call (no `;`, `&&`, `|`, or redirects):
|
|
@@ -22,6 +22,8 @@ disable-model-invocation: true
|
|
|
22
22
|
|
|
23
23
|
**Keys are the head branch.** Resolve per head: the exact head, then the longest matching family (`release/*`), then `"*"`. Each point overrides independently. An entry is `<skill> <args…>` — split on the first whitespace and pass the args verbatim.
|
|
24
24
|
|
|
25
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
26
|
+
|
|
25
27
|
## Context
|
|
26
28
|
|
|
27
29
|
Each `git`/`gh` probe is its own argv-form shell call; a non-zero exit is control flow, not something to suppress.
|
|
@@ -15,6 +15,8 @@ disable-model-invocation: true
|
|
|
15
15
|
|
|
16
16
|
**Subagent-maximal by design.** The orchestrator (the main context) must never read source files, diffs, or the tree. Every heavy read happens inside a subagent with its own isolated context window. The orchestrator only writes briefs and reads compact structured results. This is what keeps the caller's context small across many phases.
|
|
17
17
|
|
|
18
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
19
|
+
|
|
18
20
|
## Inputs
|
|
19
21
|
|
|
20
22
|
- **Work brief** — what to build or fix, concretely.
|
|
@@ -19,6 +19,8 @@ Find the root cause of a failure, get approval for the fix, fix it through the s
|
|
|
19
19
|
|
|
20
20
|
**Escalate rather than persist:** 2–3 hypotheses exhausted without confirmation, or 3 failed fix attempts, means diagnose *why* instead of trying again. One hypothesis, one change at a time — never shotgun-debug. Before escalating, re-search the prior learnings with the keywords of what you just ruled out: an exhausted hypothesis list is often a documented trap.
|
|
21
21
|
|
|
22
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
23
|
+
|
|
22
24
|
## Phase 0 — Triage
|
|
23
25
|
|
|
24
26
|
Establish the exact input, the observed symptom, the expected behavior, and how to reproduce it. If an issue/ticket is given, carry its identifier and URL through to the end. If only a stack trace or description is given, there is no issue of record — that is fine; never manufacture one.
|
|
@@ -17,6 +17,8 @@ disable-model-invocation: true
|
|
|
17
17
|
|
|
18
18
|
**Config.** Resolve `.sassywares/config.yaml` (nearest, walking up) per the `sassy-setup` skill's `references/config.md`. Handbooks land in `artifacts.handbooks`.
|
|
19
19
|
|
|
20
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
21
|
+
|
|
20
22
|
## When to run
|
|
21
23
|
|
|
22
24
|
- After a plan or multi-phase change needs a **human handover** (what landed, what is left, how to extend it).
|
|
@@ -1,55 +1,81 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sassy-handoff
|
|
3
3
|
description: >-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
4
|
+
Write a self-contained handoff file to a temp directory so a fresh session —
|
|
5
|
+
in any harness — can continue the work, then give the user its path and a
|
|
6
|
+
one-line prompt to paste. Use at a phase boundary, when context is getting
|
|
7
|
+
long, or when the user asks to hand off.
|
|
7
8
|
argument-hint: "[goal for the next session]"
|
|
8
9
|
disable-model-invocation: true
|
|
9
10
|
---
|
|
10
11
|
|
|
11
12
|
# Sassy Handoff
|
|
12
13
|
|
|
13
|
-
**Outcome:**
|
|
14
|
+
**Outcome:** a handoff file on disk that a fresh session can act on with no access to this one, plus its absolute path and a paste-ready prompt for the user. This skill never starts or replaces a session; the user does.
|
|
14
15
|
|
|
15
|
-
**
|
|
16
|
+
**Harness.** Commands you hand the user, and the temp-file rules, follow the `sassy-setup` skill's `references/harness.md`.
|
|
16
17
|
|
|
17
18
|
## Steps
|
|
18
19
|
|
|
19
20
|
1. **Determine the continuation goal.**
|
|
20
|
-
- At a sassy plan phase boundary: the next phase that is not `Done`,
|
|
21
|
-
- After a `sassy-plan` run that kept the work in a brief instead of writing a plan: continue from that
|
|
21
|
+
- At a sassy plan phase boundary: the next phase that is not `Done`, continued by invoking `sassy-work <plan> phase:<n>`.
|
|
22
|
+
- After a `sassy-plan` run that kept the work in a brief instead of writing a plan: continue from that brief. There is no plan path, so the brief itself — goal, decisions, constraints, done-when — goes in the file.
|
|
22
23
|
- Otherwise: the user's stated objective for the next session.
|
|
23
24
|
- If the goal is genuinely unclear, ask. Do not fabricate a handoff.
|
|
24
25
|
|
|
25
|
-
2. **Save state before
|
|
26
|
+
2. **Save state before writing.** The new session sees only the file, so:
|
|
26
27
|
- Update the plan's Phase tracking (and any other source-of-truth file).
|
|
27
28
|
- Confirm nothing that matters exists only in this conversation.
|
|
28
29
|
- Redact secrets, credentials, and unrelated personal data.
|
|
29
30
|
|
|
30
|
-
3. **
|
|
31
|
+
3. **Write the file.** One file per handoff, under the harness's handoff directory:
|
|
31
32
|
|
|
32
|
-
```
|
|
33
|
-
/handoff
|
|
33
|
+
```bash
|
|
34
|
+
DIR="/tmp/sassy-handoff-$(id -u)/<harness>" # pi, claude, … — the harness you are running in
|
|
35
|
+
[ -L "/tmp/sassy-handoff-$(id -u)" ] && { echo "unsafe symlink" >&2; exit 1; }
|
|
36
|
+
(umask 077; mkdir -p "$DIR")
|
|
37
|
+
FILE="$DIR/$(date -u +%Y%m%dT%H%M%SZ)-<kebab-goal>.md"
|
|
34
38
|
```
|
|
35
39
|
|
|
36
|
-
|
|
40
|
+
Use this shape, and keep every section short:
|
|
37
41
|
|
|
38
|
-
```
|
|
39
|
-
|
|
42
|
+
```md
|
|
43
|
+
# Handoff: <goal in one line>
|
|
44
|
+
|
|
45
|
+
- **Repo / cwd:** <absolute path> · **Branch:** <branch> @ <short sha> · **Written:** <ISO time>
|
|
46
|
+
- **Plan:** <plan path + next phase, or "none — brief below">
|
|
47
|
+
|
|
48
|
+
## Goal
|
|
49
|
+
What the next session must achieve, and its done-when.
|
|
50
|
+
|
|
51
|
+
## State
|
|
52
|
+
What is done, what is in progress, what is untouched. Uncommitted work, if any, named by path.
|
|
53
|
+
|
|
54
|
+
## Decisions
|
|
55
|
+
Settled choices (`D1: …`) with the reason, so the next session does not reopen them.
|
|
56
|
+
|
|
57
|
+
## Key files
|
|
58
|
+
`path:line` — why it matters. Pointers, not contents.
|
|
59
|
+
|
|
60
|
+
## Next steps
|
|
61
|
+
1. The first concrete action, including the exact skill invocation when one applies.
|
|
62
|
+
|
|
63
|
+
## Open questions and risks
|
|
64
|
+
Anything unresolved, and anything the next session must not do.
|
|
40
65
|
```
|
|
41
66
|
|
|
42
|
-
4. **
|
|
67
|
+
4. **Hand it to the user.** Report the absolute path, then one fenced, copyable prompt for the fresh session:
|
|
43
68
|
|
|
44
|
-
|
|
69
|
+
```text
|
|
70
|
+
Read <absolute path> and continue from it.
|
|
71
|
+
```
|
|
45
72
|
|
|
46
|
-
|
|
47
|
-
- **No duplicate doc.** The extension carries the context into the new session; do not also write a handoff markdown unless the user asks for one.
|
|
48
|
-
- **No plan file.** When `sassy-plan` kept the work as an in-context brief instead of writing a plan, the fresh session cannot see this conversation or point at a plan, so include the compact brief in the handoff goal.
|
|
49
|
-
- **Do not invent a continuation.** If the next step is a real fork with mutually exclusive options, present them; do not silently pick.
|
|
50
|
-
- **The skill does not switch sessions.** Never claim it did; the user runs `/handoff`.
|
|
73
|
+
Tell them to start a fresh session in their harness and paste it. When the next step is a skill, the file's **Next steps** names the invocation in that harness's syntax.
|
|
51
74
|
|
|
52
|
-
##
|
|
75
|
+
## Rules
|
|
53
76
|
|
|
54
|
-
-
|
|
55
|
-
-
|
|
77
|
+
- **Pointer-first.** Name the plan path, phase, key files, and decisions rather than reproducing them. The plan is the source of truth; the file points at it.
|
|
78
|
+
- **Self-contained.** The fresh session cannot see this conversation. When there is no plan file, the brief goes in the handoff file in full.
|
|
79
|
+
- **Outside the repo.** The file lives in the temp directory only — never committed, never inside the working tree. `/tmp` clearing on reboot is the intended lifetime.
|
|
80
|
+
- **Do not invent a continuation.** If the next step is a real fork with mutually exclusive options, list them under **Next steps**; do not silently pick.
|
|
81
|
+
- **The skill does not switch sessions.** Never claim it did; the user starts the fresh session.
|
|
@@ -16,6 +16,8 @@ disable-model-invocation: true
|
|
|
16
16
|
|
|
17
17
|
**Config.** Resolve `.sassywares/config.yaml` (nearest, walking up) per the `sassy-setup` skill's `references/config.md`. Precedence: explicit token > session instruction > config > skill default. Plans live under `artifacts.plans`; `git.repo` decides how the close-out persists; `commit.branch` decides whether the run needs its own branch.
|
|
18
18
|
|
|
19
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
20
|
+
|
|
19
21
|
## Input
|
|
20
22
|
|
|
21
23
|
Detect the shape from the argument:
|
|
@@ -46,7 +48,7 @@ A brief has no QA table, so its gate is code approval alone. Never invent, enabl
|
|
|
46
48
|
4. **The gate.** After the last phase, report: the plan path, each phase's status, what changed per phase, and the validator/reviewer summaries. Then give every phase's QA checklist verbatim — row ID, check, how to run it, what passing looks like — with unreported rows `Pending`. Ask for both decisions: code approval and the QA result per row. Wait. Do not commit.
|
|
47
49
|
5. **Resolve the gate.** Three outcomes:
|
|
48
50
|
- **Code approved and every QA row passes** → mark the rows `Done` with the user's notes, mark the phases `Done`, then run the close-out below.
|
|
49
|
-
- **Code approved, QA still pending** → tick nothing, mark nothing `Done`, commit nothing, ship nothing. Report which rows are outstanding and how to resume (
|
|
51
|
+
- **Code approved, QA still pending** → tick nothing, mark nothing `Done`, commit nothing, ship nothing. Report which rows are outstanding and how to resume (invoke `sassy-lfg <plan>` in the harness's syntax), and stop.
|
|
50
52
|
- **Any QA row fails** → run one more `sassy-cycle` on that phase with the failing rows as the intent reference, then present the gate again. A row that still fails is `Blocked` with the reason and stops the run.
|
|
51
53
|
6. **Close-out.** On both gates passing, load the `sassy-ship` skill and follow it. Do only the close-out steps the user approves. Babysit options (`merge:on|off`, `delete-branch:on|off`) are forwarded through `sassy-ship` to `sassy-babysit`. The branch from step 2 already carries the work, so ship commits and opens the PR on it; ship never creates the branch.
|
|
52
54
|
|
|
@@ -17,6 +17,8 @@ disable-model-invocation: true
|
|
|
17
17
|
|
|
18
18
|
**Config.** Resolve `.sassywares/config.yaml` (nearest, walking up) per the `sassy-setup` skill's `references/config.md`. Precedence: explicit token > session instruction > config > skill default. Plans land in `artifacts.plans`.
|
|
19
19
|
|
|
20
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
21
|
+
|
|
20
22
|
## Phase A — Discuss (brainstorm)
|
|
21
23
|
|
|
22
24
|
Have one focused discussion to turn the idea into something plannable.
|
|
@@ -15,6 +15,8 @@ disable-model-invocation: true
|
|
|
15
15
|
|
|
16
16
|
**Orchestrator only.** Never edit code yourself. The `reviewer` finds; the `worker` fixes.
|
|
17
17
|
|
|
18
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
19
|
+
|
|
18
20
|
## Review dimensions
|
|
19
21
|
|
|
20
22
|
The `reviewer` covers all four. Do not drop any of them.
|
|
@@ -15,6 +15,8 @@ disable-model-invocation: true
|
|
|
15
15
|
|
|
16
16
|
**Orchestrator only.** Do not read the repo yourself. One `scout` dispatch per independent area, in parallel when the areas do not overlap.
|
|
17
17
|
|
|
18
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
19
|
+
|
|
18
20
|
## When to use
|
|
19
21
|
|
|
20
22
|
- Before planning a change, to ground files, patterns, prior art, and any prior learning already written about the area.
|
|
@@ -34,7 +36,7 @@ disable-model-invocation: true
|
|
|
34
36
|
The scout writes it under a per-user temp directory, keyed by the worktree root:
|
|
35
37
|
|
|
36
38
|
```bash
|
|
37
|
-
ROOT="/tmp/
|
|
39
|
+
ROOT="/tmp/sassy-grounding-$(id -u)"
|
|
38
40
|
ROOT_ABS=$(cd "$(git rev-parse --show-toplevel 2>/dev/null || echo "$PWD")" && pwd -P)
|
|
39
41
|
KEY=$(printf '%s' "$ROOT_ABS" | shasum -a 256 | cut -d' ' -f1)
|
|
40
42
|
BRIEF="$ROOT/$KEY.md"
|
|
@@ -20,6 +20,8 @@ disable-model-invocation: true
|
|
|
20
20
|
|
|
21
21
|
**Never a repo by force.** Setup records topology, it does not change it. No `git init`, ever.
|
|
22
22
|
|
|
23
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow `references/harness.md`; the agent wiring in step 5 uses its tables.
|
|
24
|
+
|
|
23
25
|
## Modes
|
|
24
26
|
|
|
25
27
|
| Mode | What it does |
|
|
@@ -17,7 +17,7 @@ Refreshing the suite is this project's decision — nothing here updates itself.
|
|
|
17
17
|
|
|
18
18
|
The vendored skills and the lock are committed; `.claude/skills` is a link and is not.
|
|
19
19
|
|
|
20
|
-
`.agents/skills/` is read by pi and Cursor; Claude Code reads `.claude/skills/` only, which is why that symlink exists. After any change, `/reload`
|
|
20
|
+
`.agents/skills/` is read by pi and Cursor; Claude Code reads `.claude/skills/` only, which is why that symlink exists. After any change, reload your harness (`/reload` in pi, a new session in Claude Code).
|
|
21
21
|
|
|
22
22
|
## Agents
|
|
23
23
|
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Harness conventions
|
|
2
|
+
|
|
3
|
+
The suite runs in any harness that loads Agent Skills — pi, Claude Code, and others. No skill assumes one. When a skill says to load a skill, dispatch an agent, or hand the user a command, this file is how.
|
|
4
|
+
|
|
5
|
+
## Loading a skill
|
|
6
|
+
|
|
7
|
+
"Load the `sassy-x` skill and follow it" means: read `sassy-x/SKILL.md` from the same skills directory as the skill you are running (its sibling, `../sassy-x/SKILL.md`), then follow it. "The `sassy-x` skill's `references/y.md`" is `../sassy-x/references/y.md`.
|
|
8
|
+
|
|
9
|
+
Read the file; do not rely on the harness's skill-invocation tool. Suite skills set `disable-model-invocation: true`, and some harnesses (Claude Code) enforce that by hiding them from the model entirely. Reading the sibling file works everywhere.
|
|
10
|
+
|
|
11
|
+
A missing sibling means the suite is incomplete: stop and point at `sassy-setup`.
|
|
12
|
+
|
|
13
|
+
## Dispatching an agent
|
|
14
|
+
|
|
15
|
+
"Dispatch a `worker`" (or `scout`, `reviewer`, `validator`) means: start that [sassy-agents](https://github.com/sassywares/sassy-agents) subagent by name through the harness's own mechanism — the `subagent` tool in pi, the Agent tool with that agent type in Claude Code. "In parallel" means several dispatches in one turn.
|
|
16
|
+
|
|
17
|
+
- **Agent not installed** — stop and point at `sassy-setup`, which installs the roster for the running harness. Never substitute a generic agent silently.
|
|
18
|
+
- **Cannot dispatch** — you are yourself a subagent and the harness does not let subagents dispatch (Claude Code), or the harness has no subagents at all. Do that step inline with the same brief and constraints, and say in your result that it ran inline. Never skip it.
|
|
19
|
+
|
|
20
|
+
## Commands the user types
|
|
21
|
+
|
|
22
|
+
Skills tell the user what to run next. Write the command for the harness you are running in:
|
|
23
|
+
|
|
24
|
+
| Harness | Invoke a skill | Pick up new or changed skills and agents |
|
|
25
|
+
|---|---|---|
|
|
26
|
+
| pi | `/skill:<name> <args>` | `/reload` |
|
|
27
|
+
| Claude Code | `/<name> <args>` | restart the session |
|
|
28
|
+
|
|
29
|
+
For any other harness, give the skill name and its arguments and let the user invoke it their way.
|
|
30
|
+
|
|
31
|
+
## Temp files
|
|
32
|
+
|
|
33
|
+
Per-user temp files live under `/tmp/sassy-<purpose>-$(id -u)/`, created `0700`, never inside the repo. A symlinked root aborts rather than being followed. Two use it today: the grounding brief (`/tmp/sassy-grounding-<uid>/`, written by `scout`) and handoff files (`/tmp/sassy-handoff-<uid>/<harness>/`, written by `sassy-handoff`).
|
|
@@ -20,6 +20,8 @@ disable-model-invocation: true
|
|
|
20
20
|
|
|
21
21
|
**Subagent-maximal by design.** The orchestrator delegates every heavy step (docs, tree reads, git, gh) to subagents and only receives compact summaries. It never reads diffs, file contents, or the tree itself.
|
|
22
22
|
|
|
23
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
24
|
+
|
|
23
25
|
## Steps (in order)
|
|
24
26
|
|
|
25
27
|
1. **Review gate.** In `mode:manual` only: present a compact shipping summary — the plan or topic, the changes from the cycle results, the docs the close-out expects to write, the branch/PR target, the babysit options, and **the hooks resolved for this run's head — point, skill, and args — or that none are declared** — and wait. In `mode:auto`, skip this gate.
|
|
@@ -20,6 +20,8 @@ disable-model-invocation: true
|
|
|
20
20
|
|
|
21
21
|
**Scope:** skills only — not extensions, packages, or prompt templates.
|
|
22
22
|
|
|
23
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
24
|
+
|
|
23
25
|
## One mode, one reference
|
|
24
26
|
|
|
25
27
|
Each mode's mechanics live in its own file. Read the one you were dispatched to — **and only that one**. A `create` never reads `retune`'s failure map, and a `verify` never picks up a fix from it: that adjacency is exactly how a read-only check turns into an edit.
|
|
@@ -30,18 +32,21 @@ Each mode's mechanics live in its own file. Read the one you were dispatched to
|
|
|
30
32
|
| `retune` | `references/retune.md` | the smallest edit that kills one named failure, re-verified |
|
|
31
33
|
| `verify` | `references/verify.md` | a report, and no file touched |
|
|
32
34
|
|
|
33
|
-
Everything below is shared by all three: the location table,
|
|
35
|
+
Everything below is shared by all three: the location table, the Agent Skills rules, the shape rule, the checklist, and the verification run. Read it once, then your one reference.
|
|
36
|
+
|
|
37
|
+
## Agent Skills rules (the essentials)
|
|
34
38
|
|
|
35
|
-
|
|
39
|
+
The suite targets the **Agent Skills specification** (agentskills.io), so a skill works in any harness that loads it — pi, Claude Code, and others. Write to the spec, not to one harness.
|
|
36
40
|
|
|
37
41
|
- A skill is a **directory containing `SKILL.md`**. Discovered recursively.
|
|
38
42
|
- **Name:** lowercase letters, numbers, hyphens; no leading/trailing/consecutive hyphens; ≤64 chars. Match the directory name.
|
|
39
|
-
- **Description:** required, ≤1024 chars.
|
|
43
|
+
- **Description:** required, ≤1024 chars. The harness advertises name + description up front; the body loads only when the skill is used, so the description alone decides whether the skill ever fires. Write **what it does and when to use it**, with concrete verbs and the words a user would actually say — "can't trigger" is almost always a description that named the topic and never the moment.
|
|
40
44
|
- **Frontmatter fields:** `name`, `description` (required); optional `license`, `compatibility`, `metadata`, `allowed-tools`, `disable-model-invocation`.
|
|
41
|
-
- Use `disable-model-invocation: true` for skills that should run only
|
|
45
|
+
- Use `disable-model-invocation: true` for skills that should run only when the user invokes them. Some harnesses (Claude Code) then hide the skill from the model entirely, so a skill that loads another must read its `SKILL.md` — the `sassy-setup` skill's `references/harness.md` defines that.
|
|
42
46
|
- **Reference other skills by name, never by absolute path.** The suite can live in `~/.agents/skills/`, a project `.agents/skills/`, or inside `node_modules/`, so "load the sassy-commit skill" is portable where a `~/…` path is not. Name a bundled file as "the sassy-setup skill's `references/config.md`".
|
|
43
|
-
-
|
|
44
|
-
-
|
|
47
|
+
- Harnesses **validate when they load skills**; a malformed or descriptionless skill warns or is skipped. Reloading applies changes (the command per harness is in `references/harness.md`).
|
|
48
|
+
- The spec at agentskills.io is authoritative when the rules here are not enough. Never rely on a field or behavior only one harness supports.
|
|
49
|
+
- **No harness-only machinery.** No harness-specific commands, extensions, tools, or paths in the workflow. Where harnesses genuinely differ, point at `references/harness.md` instead of naming one.
|
|
45
50
|
|
|
46
51
|
## Where the skill lives
|
|
47
52
|
|
|
@@ -57,7 +62,7 @@ Recommend by audience: the team needs it → **project**; only the requester →
|
|
|
57
62
|
|
|
58
63
|
Internal is the only one that ships to everyone. Never choose it silently: once it lands on `main`, every consumer gets it on their next `npx skills update`. Say that out loud when you do.
|
|
59
64
|
|
|
60
|
-
Paths in the table are the defaults. Harnesses also read `skills` arrays in settings and installed packages, so a skill may live elsewhere — write down where it actually went.
|
|
65
|
+
Paths in the table are the defaults. `.agents/skills/` is read by pi and others; Claude Code reads only `.claude/skills/` (project) and `~/.claude/skills/` (user), so a hand-placed skill there also needs a link in those directories — `npx skills add` creates them for installed skills. Harnesses also read `skills` arrays in settings and installed packages, so a skill may live elsewhere — write down where it actually went.
|
|
61
66
|
|
|
62
67
|
## Shape rule — split on outcomes, not on topics
|
|
63
68
|
|
|
@@ -83,9 +88,9 @@ The gate for a new or edited skill, and the instrument in `mode:verify`.
|
|
|
83
88
|
|
|
84
89
|
## Verification — how every mode ends
|
|
85
90
|
|
|
86
|
-
-
|
|
87
|
-
-
|
|
88
|
-
-
|
|
91
|
+
- Reload the harness (command per `references/harness.md`).
|
|
92
|
+
- Confirm the harness lists the skill with no load warning.
|
|
93
|
+
- Invoke it with the harness's syntax and confirm it loads and behaves.
|
|
89
94
|
|
|
90
95
|
`create` and `retune` are not finished until this passes; `verify` exists to run it and report.
|
|
91
96
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Mode: create — write a new skill
|
|
2
2
|
|
|
3
|
-
Read the skill's shared sections first (locations,
|
|
3
|
+
Read the skill's shared sections first (locations, Agent Skills rules, shape rule, checklist). This file is the write side only.
|
|
4
4
|
|
|
5
5
|
## 1. Intake
|
|
6
6
|
|
|
@@ -24,7 +24,7 @@ Two paragraphs of route-specific instruction are worth their own file; a shared
|
|
|
24
24
|
|
|
25
25
|
## 3. Frontmatter
|
|
26
26
|
|
|
27
|
-
`name` matches the directory. `description` states what and when, ≤1024 chars, in the user's vocabulary. Add `disable-model-invocation: true` when the skill should only run
|
|
27
|
+
`name` matches the directory. `description` states what and when, ≤1024 chars, in the user's vocabulary. Add `disable-model-invocation: true` when the skill should only run when the user invokes it, and an `argument-hint` when it takes one.
|
|
28
28
|
|
|
29
29
|
Do not put workflow detail in the description — it decides triggering, not execution.
|
|
30
30
|
|
|
@@ -43,6 +43,6 @@ Plain English, no filler. Where the skill returns a result, give the shape of th
|
|
|
43
43
|
|
|
44
44
|
- **Internal:** `skills/<name>/` in the sassy-skills repo, named `sassy-<name>`; then `scripts/sync.sh` — it commits, bumps the patch version, tags, and publishes. Say that this ships to everyone before you do it.
|
|
45
45
|
- **Project:** `<repo>/.agents/skills/<name>/`, prefixed with the project so it cannot collide with a suite skill or a harness built-in; then commit it.
|
|
46
|
-
- **User:** `~/.agents/skills/<name
|
|
46
|
+
- **User:** `~/.agents/skills/<name>/` (plus a `~/.claude/skills/<name>` link for Claude Code). Nothing gates it, so check it actually works before trusting it.
|
|
47
47
|
|
|
48
|
-
Then run the skill's Verification section:
|
|
48
|
+
Then run the skill's Verification section: reload, no load warning, invoking it behaves. A skill that has never been invoked has not been written.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Mode: retune — the smallest fix for one named failure
|
|
2
2
|
|
|
3
|
-
Read the skill's shared sections first (locations,
|
|
3
|
+
Read the skill's shared sections first (locations, Agent Skills rules, shape rule, checklist). This file is the failure map.
|
|
4
4
|
|
|
5
5
|
## 1. Name the failure before touching anything
|
|
6
6
|
|
|
@@ -30,4 +30,4 @@ If the request names no failure, ask for one. A retune without a failure is a re
|
|
|
30
30
|
|
|
31
31
|
- Internal skill → `scripts/sync.sh` (commit, bump, tag, publish; say so before running it).
|
|
32
32
|
- Project or user skill → committed or written in place, as its location requires.
|
|
33
|
-
- Then run the skill's Verification section:
|
|
33
|
+
- Then run the skill's Verification section: reload, no load warning, invoking it behaves. A retune that was never invoked has not been verified.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Mode: verify — read, judge, report, change nothing
|
|
2
2
|
|
|
3
|
-
Read the skill's shared sections first (locations,
|
|
3
|
+
Read the skill's shared sections first (locations, Agent Skills rules, shape rule, checklist). This file is the read-only sweep.
|
|
4
4
|
|
|
5
5
|
**No writes. Not one.** No edits, no renames, no moving files between locations, no version bumps, no `scripts/sync.sh`. A verify run that "just fixes a typo while it is there" has become a retune without a named failure, and the user loses the report they asked for.
|
|
6
6
|
|
|
@@ -21,7 +21,7 @@ Walk the shared **Self-check checklist** item by item. For each, the evidence is
|
|
|
21
21
|
|
|
22
22
|
## 3. Run the skill, if it is installed
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
Reload the harness, confirm it lists the skill with no load warning, then invoke it (syntax per the `sassy-setup` skill's `references/harness.md`). Quote what it did. If it cannot be invoked here — different harness, uninstalled, needs credentials — say that plainly instead of implying a pass.
|
|
25
25
|
|
|
26
26
|
## 4. Report in one shape
|
|
27
27
|
|
|
@@ -17,6 +17,8 @@ disable-model-invocation: true
|
|
|
17
17
|
|
|
18
18
|
**This skill needs an intent.** Validation without something to validate against is meaningless. If the user gave no plan path or intent, ask for one before doing anything.
|
|
19
19
|
|
|
20
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
21
|
+
|
|
20
22
|
## Workflow
|
|
21
23
|
|
|
22
24
|
1. **Intent.** Resolve the reference: a plan path plus phase (preferred), or an explicit intent the user states. If neither exists, ask and stop.
|
|
@@ -19,6 +19,8 @@ disable-model-invocation: true
|
|
|
19
19
|
|
|
20
20
|
**Config.** Resolve `.sassywares/config.yaml` (nearest, walking up) per the `sassy-setup` skill's `references/config.md`. Precedence: explicit token > session instruction > config > skill default. `git.repo` decides what the phase commit does, `artifacts.plans` decides where the plan lives, and `commit.branch` decides whether this run needs its own branch.
|
|
21
21
|
|
|
22
|
+
**Harness.** Loading another skill, dispatching an agent, and any command you hand the user follow the `sassy-setup` skill's `references/harness.md`. Read it once per run, before the first of those.
|
|
23
|
+
|
|
22
24
|
## Input
|
|
23
25
|
|
|
24
26
|
Detect the shape from the argument:
|
|
@@ -65,17 +67,11 @@ Always gate before committing. One worker fixes findings, one re-check, then pre
|
|
|
65
67
|
|
|
66
68
|
Do not hand off or close out until the commit or snapshot succeeds.
|
|
67
69
|
8. **On commit success:**
|
|
68
|
-
- **A next phase exists** →
|
|
69
|
-
|
|
70
|
-
```text
|
|
71
|
-
/handoff Execute phase <next> of <plan>. Run /skill:sassy-work <plan> phase:<next>
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
The user submits the seeded prompt once, and the next phase starts in the fresh session.
|
|
70
|
+
- **A next phase exists** → load the `sassy-handoff` skill with the goal *Execute phase <next> of <plan> by invoking `sassy-work <plan> phase:<next>`*. It writes the handoff file and gives the user its path and a one-line prompt; the next phase starts when they paste it into a fresh session.
|
|
75
71
|
|
|
76
72
|
- **This was the last phase** → run the close-out: load the `sassy-ship` skill and follow it.
|
|
77
73
|
|
|
78
|
-
**Why a
|
|
74
|
+
**Why a fresh session:** one phase per context keeps every phase's context small. A skill cannot start a session itself, so the handoff file is the bridge and the user opens the session.
|
|
79
75
|
|
|
80
76
|
## Brief input
|
|
81
77
|
|
|
@@ -1,95 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: sassy-compare
|
|
3
|
-
description: >-
|
|
4
|
-
Run the same prompt against two models in separate git worktrees, compare
|
|
5
|
-
their final output, code changes, timing, tokens, and errors, then tear the
|
|
6
|
-
worktrees down and report. Use when choosing between two models for a task or
|
|
7
|
-
checking whether one model is faster or more reliable.
|
|
8
|
-
argument-hint: "model-a:<model> [thinking-a:<level>] model-b:<model> [thinking-b:<level>] [--base <ref>] [--timeout <sec>] [--keep] <prompt>"
|
|
9
|
-
disable-model-invocation: true
|
|
10
|
-
---
|
|
11
|
-
|
|
12
|
-
# Sassy Compare
|
|
13
|
-
|
|
14
|
-
**Outcome:** one prompt run against two models in isolated worktrees, with optional per-side thinking levels, a report comparing output, code changes, wall time, model time, tool time, turns, tokens, cost, and errors, and the worktrees removed unless `--keep` was passed.
|
|
15
|
-
|
|
16
|
-
**The main worktree is never touched.** Compare runs in disposable worktrees created from `HEAD` (or `--base`).
|
|
17
|
-
|
|
18
|
-
## Input
|
|
19
|
-
|
|
20
|
-
Parse the request into these parts:
|
|
21
|
-
|
|
22
|
-
- `model-a:<model-identifier>` — first model.
|
|
23
|
-
- `model-b:<model-identifier>` — second model.
|
|
24
|
-
- `thinking-a:<level>` — optional thinking level for model-a.
|
|
25
|
-
- `thinking-b:<level>` — optional thinking level for model-b.
|
|
26
|
-
- The prompt — everything else, verbatim.
|
|
27
|
-
|
|
28
|
-
Model identifiers are passed straight to `pi --model`; `provider/model` and an optional `:thinking` suffix are allowed, for example `deepseek/deepseek-flash:high`. Thinking levels are `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. An explicit `thinking-a`/`thinking-b` wins over a `:level` suffix.
|
|
29
|
-
|
|
30
|
-
Use the thinking tokens to compare the same model at different effort levels, or two different agents/models with different effort:
|
|
31
|
-
|
|
32
|
-
```text
|
|
33
|
-
/sassy-compare model-a:deepseek/deepseek-flash thinking-a:low model-b:deepseek/deepseek-flash thinking-b:high "Implement the pagination helper"
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
If a model or the prompt is missing, ask one short question for the missing part. Do not guess a model.
|
|
37
|
-
|
|
38
|
-
## Workflow
|
|
39
|
-
|
|
40
|
-
1. **Resolve the skill directory.** Use the absolute path the host supplied for this `SKILL.md`; the script is at `scripts/compare.py` beside it.
|
|
41
|
-
2. **Run the comparison.** Pass the prompt as one argument:
|
|
42
|
-
|
|
43
|
-
```bash
|
|
44
|
-
python3 <skill-dir>/scripts/compare.py \
|
|
45
|
-
--model-a "<model-a>" \
|
|
46
|
-
--model-b "<model-b>" \
|
|
47
|
-
--prompt "<prompt>"
|
|
48
|
-
|
|
49
|
-
# Add only when the request supplied them:
|
|
50
|
-
# --thinking-a "<level>" --thinking-b "<level>"
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
Useful options:
|
|
54
|
-
|
|
55
|
-
- `--thinking-a <level>` — thinking level for model-a; omit to use the model's default.
|
|
56
|
-
- `--thinking-b <level>` — thinking level for model-b; omit to use the model's default.
|
|
57
|
-
- `--base <ref>` — branch both worktrees from a ref other than `HEAD`.
|
|
58
|
-
- `--timeout <seconds>` — per-model timeout, default 1800.
|
|
59
|
-
- `--keep` — keep the worktrees after the run. Use only when the user asks to inspect them.
|
|
60
|
-
- `--prompt-file <path>` — read the prompt from a file instead of inline.
|
|
61
|
-
- `--dry-run` — create and tear down the worktrees without invoking a model.
|
|
62
|
-
|
|
63
|
-
3. **Read the report.** The script prints the report path, normally under `~/.pi/compare/<run-id>/report.md`. Read it and present:
|
|
64
|
-
- the summary table,
|
|
65
|
-
- the final output diff,
|
|
66
|
-
- the code diff, or the file lists when the diff is too large,
|
|
67
|
-
- any errors or timeouts.
|
|
68
|
-
|
|
69
|
-
4. **Report facts, not a verdict.** Do not declare a winner unless the user asked for a recommendation. If asked, base the recommendation on the evidence in the report and name the tradeoff (quality, time, cost, reliability).
|
|
70
|
-
|
|
71
|
-
## What the script does
|
|
72
|
-
|
|
73
|
-
- Creates `model-a` and `model-b` worktrees from the chosen base commit.
|
|
74
|
-
- Runs each model sequentially with `pi --mode json --no-session --model <model>` and the requested `--thinking` level, so timing is not polluted by the two runs competing for resources.
|
|
75
|
-
- Captures the final assistant text, the full diff against the base (committed and uncommitted), file lists, stderr, turns, model time, tool time, token usage, cost, retries, and error turns.
|
|
76
|
-
- Writes the report and artifacts under `~/.pi/compare/<run-id>/`.
|
|
77
|
-
- Removes both worktrees and prunes git worktree metadata, unless `--keep` is set.
|
|
78
|
-
|
|
79
|
-
## Hard rules
|
|
80
|
-
|
|
81
|
-
| Rule | Requirement |
|
|
82
|
-
|------|-------------|
|
|
83
|
-
| Isolated worktrees | Never run the prompt in the main worktree. Both models get their own worktree. |
|
|
84
|
-
| Same prompt and base | Both models receive the identical prompt and start from the same commit. |
|
|
85
|
-
| Sequential runs | Do not run the two models in parallel by default; concurrent runs distort wall-time and provider-latency comparisons. |
|
|
86
|
-
| No false comparison | If one run times out or fails, report the failure and do not compare output quality. |
|
|
87
|
-
| No silent cleanup loss | Worktrees are removed by default; artifacts and the report stay under `~/.pi/compare/`. |
|
|
88
|
-
| Uncommitted work is excluded | Worktrees start from the base commit. Warn the user when the repo is dirty; do not copy local changes. |
|
|
89
|
-
|
|
90
|
-
## Failure handling
|
|
91
|
-
|
|
92
|
-
- `pi` is missing or not runnable → stop and report the binary path that failed.
|
|
93
|
-
- Not inside a git repository, or the base ref is invalid → stop with the exact error.
|
|
94
|
-
- A model times out → report the timeout, keep that model's partial output and stderr, and do not compare quality.
|
|
95
|
-
- `git worktree add` fails → stop, report the command output, and remove anything already created.
|
|
@@ -1,493 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env python3
|
|
2
|
-
"""Run one prompt against two models in isolated git worktrees and compare."""
|
|
3
|
-
|
|
4
|
-
from __future__ import annotations
|
|
5
|
-
|
|
6
|
-
import argparse
|
|
7
|
-
import json
|
|
8
|
-
import os
|
|
9
|
-
import shutil
|
|
10
|
-
import signal
|
|
11
|
-
import subprocess
|
|
12
|
-
import sys
|
|
13
|
-
import time
|
|
14
|
-
from difflib import unified_diff
|
|
15
|
-
from pathlib import Path
|
|
16
|
-
|
|
17
|
-
REPORT_DIFF_LINES = 400
|
|
18
|
-
REPORT_DIFF_CHARS = 60_000
|
|
19
|
-
THINKING_LEVELS = ("off", "minimal", "low", "medium", "high", "xhigh", "max")
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
def git(args: list[str], cwd: Path, check: bool = True) -> subprocess.CompletedProcess[str]:
|
|
23
|
-
return subprocess.run(["git", *args], cwd=cwd, text=True, capture_output=True, check=check)
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
def git_text(args: list[str], cwd: Path) -> str:
|
|
27
|
-
return git(args, cwd).stdout.strip()
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
def truncate(text: str, lines: int = REPORT_DIFF_LINES, chars: int = REPORT_DIFF_CHARS) -> str:
|
|
31
|
-
if not text:
|
|
32
|
-
return ""
|
|
33
|
-
out: list[str] = []
|
|
34
|
-
used = 0
|
|
35
|
-
for line in text.splitlines():
|
|
36
|
-
if len(out) >= lines or used + len(line) + 1 > chars:
|
|
37
|
-
out.append("\n[... truncated; see the full artifact file ...]")
|
|
38
|
-
break
|
|
39
|
-
out.append(line)
|
|
40
|
-
used += len(line) + 1
|
|
41
|
-
return "\n".join(out)
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
def thinking_from_model(model: str) -> str | None:
|
|
45
|
-
if ":" not in model:
|
|
46
|
-
return None
|
|
47
|
-
suffix = model.rsplit(":", 1)[1]
|
|
48
|
-
return suffix if suffix in THINKING_LEVELS else None
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
def fmt_ms(ms: int) -> str:
|
|
52
|
-
if ms < 60_000:
|
|
53
|
-
return f"{ms / 1000:.1f}s"
|
|
54
|
-
return f"{ms / 60_000:.1f}m"
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
def parse_events(path: Path) -> dict:
|
|
58
|
-
out = {
|
|
59
|
-
"turns": 0,
|
|
60
|
-
"model_ms": 0,
|
|
61
|
-
"tool_ms": 0,
|
|
62
|
-
"errors": 0,
|
|
63
|
-
"retries": 0,
|
|
64
|
-
"input_tokens": 0,
|
|
65
|
-
"output_tokens": 0,
|
|
66
|
-
"cache_read_tokens": 0,
|
|
67
|
-
"cache_write_tokens": 0,
|
|
68
|
-
"reasoning_tokens": 0,
|
|
69
|
-
"cost": 0.0,
|
|
70
|
-
"aborted": False,
|
|
71
|
-
"stop_reason": None,
|
|
72
|
-
"actual_model": None,
|
|
73
|
-
"output_text": "",
|
|
74
|
-
}
|
|
75
|
-
last_text = ""
|
|
76
|
-
saw_tool_execution_end = False
|
|
77
|
-
tool_result_ms = 0
|
|
78
|
-
tool_execution_ms = 0
|
|
79
|
-
|
|
80
|
-
if not path.exists():
|
|
81
|
-
return out
|
|
82
|
-
|
|
83
|
-
for line in path.read_text(errors="replace").splitlines():
|
|
84
|
-
try:
|
|
85
|
-
event = json.loads(line)
|
|
86
|
-
except json.JSONDecodeError:
|
|
87
|
-
continue
|
|
88
|
-
kind = event.get("type")
|
|
89
|
-
|
|
90
|
-
if kind == "message_end":
|
|
91
|
-
message = event.get("message") or {}
|
|
92
|
-
role = message.get("role")
|
|
93
|
-
if role == "assistant":
|
|
94
|
-
out["turns"] += 1
|
|
95
|
-
out["model_ms"] += int(message.get("durationMs") or 0)
|
|
96
|
-
usage = message.get("usage") or {}
|
|
97
|
-
out["input_tokens"] += int(usage.get("input") or 0)
|
|
98
|
-
out["output_tokens"] += int(usage.get("output") or 0)
|
|
99
|
-
out["cache_read_tokens"] += int(usage.get("cacheRead") or 0)
|
|
100
|
-
out["cache_write_tokens"] += int(usage.get("cacheWrite") or 0)
|
|
101
|
-
out["reasoning_tokens"] += int(usage.get("reasoning") or 0)
|
|
102
|
-
cost = usage.get("cost")
|
|
103
|
-
if isinstance(cost, dict):
|
|
104
|
-
out["cost"] += float(cost.get("total") or 0)
|
|
105
|
-
elif isinstance(cost, (int, float)):
|
|
106
|
-
out["cost"] += float(cost)
|
|
107
|
-
if message.get("stopReason") == "error" or message.get("errorMessage"):
|
|
108
|
-
out["errors"] += 1
|
|
109
|
-
if message.get("stopReason"):
|
|
110
|
-
out["stop_reason"] = message.get("stopReason")
|
|
111
|
-
if message.get("model"):
|
|
112
|
-
out["actual_model"] = f"{message.get('provider', '')}/{message.get('model')}".strip("/")
|
|
113
|
-
text = "".join(
|
|
114
|
-
part.get("text", "")
|
|
115
|
-
for part in message.get("content", [])
|
|
116
|
-
if isinstance(part, dict) and part.get("type") == "text"
|
|
117
|
-
)
|
|
118
|
-
if text:
|
|
119
|
-
last_text = text
|
|
120
|
-
elif role == "toolResult":
|
|
121
|
-
tool_result_ms += int(message.get("durationMs") or 0)
|
|
122
|
-
|
|
123
|
-
elif kind == "tool_execution_end":
|
|
124
|
-
saw_tool_execution_end = True
|
|
125
|
-
tool_execution_ms += int(event.get("durationMs") or 0)
|
|
126
|
-
elif kind == "auto_retry_start":
|
|
127
|
-
out["retries"] += 1
|
|
128
|
-
elif kind == "agent_settled":
|
|
129
|
-
out["aborted"] = bool(event.get("aborted"))
|
|
130
|
-
elif kind == "agent_end" and not last_text:
|
|
131
|
-
for message in reversed(event.get("messages") or []):
|
|
132
|
-
if message.get("role") == "assistant":
|
|
133
|
-
text = "".join(
|
|
134
|
-
part.get("text", "")
|
|
135
|
-
for part in message.get("content", [])
|
|
136
|
-
if isinstance(part, dict) and part.get("type") == "text"
|
|
137
|
-
)
|
|
138
|
-
if text:
|
|
139
|
-
last_text = text
|
|
140
|
-
break
|
|
141
|
-
|
|
142
|
-
out["tool_ms"] = tool_execution_ms if saw_tool_execution_end else tool_result_ms
|
|
143
|
-
out["output_text"] = last_text
|
|
144
|
-
return out
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
def run_model(
|
|
148
|
-
label: str,
|
|
149
|
-
model: str,
|
|
150
|
-
thinking: str | None,
|
|
151
|
-
worktree: Path,
|
|
152
|
-
out_dir: Path,
|
|
153
|
-
prompt: str,
|
|
154
|
-
args,
|
|
155
|
-
) -> dict:
|
|
156
|
-
events_path = out_dir / "events.jsonl"
|
|
157
|
-
stderr_path = out_dir / "stderr.txt"
|
|
158
|
-
cmd = [args.pi_bin, "--mode", "json", "--no-session"]
|
|
159
|
-
if thinking:
|
|
160
|
-
cmd += ["--thinking", thinking]
|
|
161
|
-
cmd += ["--model", model, "--", prompt]
|
|
162
|
-
|
|
163
|
-
started = time.time()
|
|
164
|
-
timed_out = False
|
|
165
|
-
exit_code = -1
|
|
166
|
-
with events_path.open("w") as events_file, stderr_path.open("w") as stderr_file:
|
|
167
|
-
proc = subprocess.Popen(
|
|
168
|
-
cmd,
|
|
169
|
-
cwd=worktree,
|
|
170
|
-
stdout=events_file,
|
|
171
|
-
stderr=stderr_file,
|
|
172
|
-
text=True,
|
|
173
|
-
start_new_session=True,
|
|
174
|
-
)
|
|
175
|
-
try:
|
|
176
|
-
exit_code = proc.wait(timeout=args.timeout)
|
|
177
|
-
except subprocess.TimeoutExpired:
|
|
178
|
-
timed_out = True
|
|
179
|
-
try:
|
|
180
|
-
os.killpg(proc.pid, signal.SIGTERM)
|
|
181
|
-
except ProcessLookupError:
|
|
182
|
-
pass
|
|
183
|
-
try:
|
|
184
|
-
proc.wait(timeout=5)
|
|
185
|
-
except subprocess.TimeoutExpired:
|
|
186
|
-
try:
|
|
187
|
-
os.killpg(proc.pid, signal.SIGKILL)
|
|
188
|
-
except ProcessLookupError:
|
|
189
|
-
pass
|
|
190
|
-
proc.wait()
|
|
191
|
-
exit_code = proc.returncode if proc.returncode is not None else -1
|
|
192
|
-
|
|
193
|
-
wall_ms = int((time.time() - started) * 1000)
|
|
194
|
-
summary = parse_events(events_path)
|
|
195
|
-
total_cost = summary.pop("cost")
|
|
196
|
-
output_text = summary.pop("output_text")
|
|
197
|
-
(out_dir / "output.txt").write_text(output_text + "\n")
|
|
198
|
-
|
|
199
|
-
# Capture committed and uncommitted changes together. `-N` makes untracked
|
|
200
|
-
# files visible to `git diff` without staging their contents.
|
|
201
|
-
git(["add", "-A", "-N"], cwd=worktree, check=False)
|
|
202
|
-
changes = git(["diff", "--binary", args.base_sha], cwd=worktree, check=False).stdout
|
|
203
|
-
(out_dir / "changes.diff").write_text(changes)
|
|
204
|
-
(out_dir / "changes.stat").write_text(
|
|
205
|
-
git(["diff", "--stat", args.base_sha], cwd=worktree, check=False).stdout
|
|
206
|
-
)
|
|
207
|
-
(out_dir / "files.txt").write_text(
|
|
208
|
-
git(["diff", "--name-status", args.base_sha], cwd=worktree, check=False).stdout
|
|
209
|
-
)
|
|
210
|
-
(out_dir / "status.txt").write_text(
|
|
211
|
-
git(["status", "--porcelain"], cwd=worktree, check=False).stdout
|
|
212
|
-
)
|
|
213
|
-
(out_dir / "commits.txt").write_text(
|
|
214
|
-
git(["log", "--oneline", f"{args.base_sha}..HEAD"], cwd=worktree, check=False).stdout
|
|
215
|
-
)
|
|
216
|
-
|
|
217
|
-
stderr_text = stderr_path.read_text(errors="replace") if stderr_path.exists() else ""
|
|
218
|
-
summary.update(
|
|
219
|
-
{
|
|
220
|
-
"label": label,
|
|
221
|
-
"requested_model": model,
|
|
222
|
-
"thinking": thinking,
|
|
223
|
-
"wall_ms": wall_ms,
|
|
224
|
-
"exit_code": exit_code,
|
|
225
|
-
"timed_out": timed_out,
|
|
226
|
-
"cost": total_cost,
|
|
227
|
-
"stderr_tail": stderr_text[-4000:],
|
|
228
|
-
}
|
|
229
|
-
)
|
|
230
|
-
(out_dir / "summary.json").write_text(json.dumps(summary, indent=2) + "\n")
|
|
231
|
-
return summary
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
def write_report(
|
|
235
|
-
run_root: Path,
|
|
236
|
-
model_a: str,
|
|
237
|
-
model_b: str,
|
|
238
|
-
thinking_a: str | None,
|
|
239
|
-
thinking_b: str | None,
|
|
240
|
-
prompt: str,
|
|
241
|
-
base: str,
|
|
242
|
-
worktree_a: Path,
|
|
243
|
-
worktree_b: Path,
|
|
244
|
-
summary_a: dict,
|
|
245
|
-
summary_b: dict,
|
|
246
|
-
dry_run: bool,
|
|
247
|
-
) -> Path:
|
|
248
|
-
lines: list[str] = []
|
|
249
|
-
lines.append("# sassy-compare report")
|
|
250
|
-
lines.append("")
|
|
251
|
-
lines.append(f"- Base: `{base}`")
|
|
252
|
-
lines.append(f"- model-a: `{model_a}` (thinking: `{thinking_a or 'model default'}`)")
|
|
253
|
-
lines.append(f"- model-b: `{model_b}` (thinking: `{thinking_b or 'model default'}`)")
|
|
254
|
-
lines.append(f"- Prompt: {prompt.strip()[:2000]}")
|
|
255
|
-
lines.append("")
|
|
256
|
-
|
|
257
|
-
if dry_run:
|
|
258
|
-
lines.append("Dry run only: no model was invoked.")
|
|
259
|
-
lines.append("")
|
|
260
|
-
|
|
261
|
-
header = ["Metric", f"model-a ({model_a})", f"model-b ({model_b})"]
|
|
262
|
-
rows = [
|
|
263
|
-
("Thinking level", thinking_a or "model default", thinking_b or "model default"),
|
|
264
|
-
("Wall time", fmt_ms(summary_a.get("wall_ms", 0)), fmt_ms(summary_b.get("wall_ms", 0))),
|
|
265
|
-
("Exit code", str(summary_a.get("exit_code")), str(summary_b.get("exit_code"))),
|
|
266
|
-
("Timed out", "yes" if summary_a.get("timed_out") else "no", "yes" if summary_b.get("timed_out") else "no"),
|
|
267
|
-
("Turns", str(summary_a.get("turns", 0)), str(summary_b.get("turns", 0))),
|
|
268
|
-
("Model time", fmt_ms(summary_a.get("model_ms", 0)), fmt_ms(summary_b.get("model_ms", 0))),
|
|
269
|
-
("Tool time", fmt_ms(summary_a.get("tool_ms", 0)), fmt_ms(summary_b.get("tool_ms", 0))),
|
|
270
|
-
("Output tokens", str(summary_a.get("output_tokens", 0)), str(summary_b.get("output_tokens", 0))),
|
|
271
|
-
("Reasoning tokens", str(summary_a.get("reasoning_tokens", 0)), str(summary_b.get("reasoning_tokens", 0))),
|
|
272
|
-
("Cost", f"${summary_a.get('cost', 0):.4f}", f"${summary_b.get('cost', 0):.4f}"),
|
|
273
|
-
("Error turns", str(summary_a.get("errors", 0)), str(summary_b.get("errors", 0))),
|
|
274
|
-
("Retries", str(summary_a.get("retries", 0)), str(summary_b.get("retries", 0))),
|
|
275
|
-
("Aborted", "yes" if summary_a.get("aborted") else "no", "yes" if summary_b.get("aborted") else "no"),
|
|
276
|
-
]
|
|
277
|
-
lines.append("## Summary")
|
|
278
|
-
lines.append("")
|
|
279
|
-
lines.append("| " + " | ".join(header) + " |")
|
|
280
|
-
lines.append("|" + "|".join(["---"] * len(header)) + "|")
|
|
281
|
-
for row in rows:
|
|
282
|
-
lines.append("| " + " | ".join(row) + " |")
|
|
283
|
-
lines.append("")
|
|
284
|
-
|
|
285
|
-
for label, summary in (("model-a", summary_a), ("model-b", summary_b)):
|
|
286
|
-
lines.append(f"### {label} files")
|
|
287
|
-
lines.append("")
|
|
288
|
-
files = (run_root / "artifacts" / label / "files.txt").read_text(errors="replace").strip()
|
|
289
|
-
lines.append("```text")
|
|
290
|
-
lines.append(files or "(no file changes)")
|
|
291
|
-
lines.append("```")
|
|
292
|
-
if summary.get("stderr_tail", "").strip():
|
|
293
|
-
lines.append("")
|
|
294
|
-
lines.append(f"### {label} stderr tail")
|
|
295
|
-
lines.append("")
|
|
296
|
-
lines.append("```text")
|
|
297
|
-
lines.append(truncate(summary["stderr_tail"], lines=80, chars=8000))
|
|
298
|
-
lines.append("```")
|
|
299
|
-
lines.append("")
|
|
300
|
-
|
|
301
|
-
output_a = (run_root / "artifacts" / "model-a" / "output.txt").read_text(errors="replace")
|
|
302
|
-
output_b = (run_root / "artifacts" / "model-b" / "output.txt").read_text(errors="replace")
|
|
303
|
-
output_diff = "".join(
|
|
304
|
-
unified_diff(
|
|
305
|
-
output_a.splitlines(keepends=True),
|
|
306
|
-
output_b.splitlines(keepends=True),
|
|
307
|
-
fromfile="model-a/output.txt",
|
|
308
|
-
tofile="model-b/output.txt",
|
|
309
|
-
)
|
|
310
|
-
)
|
|
311
|
-
(run_root / "output.diff").write_text(output_diff)
|
|
312
|
-
lines.append("## Final output diff")
|
|
313
|
-
lines.append("")
|
|
314
|
-
if output_diff:
|
|
315
|
-
lines.append("```diff")
|
|
316
|
-
lines.append(truncate(output_diff))
|
|
317
|
-
lines.append("```")
|
|
318
|
-
else:
|
|
319
|
-
lines.append("No difference in final assistant text.")
|
|
320
|
-
lines.append("")
|
|
321
|
-
|
|
322
|
-
code_diff = subprocess.run(
|
|
323
|
-
["diff", "-ruN", "--exclude=.git", str(worktree_a), str(worktree_b)],
|
|
324
|
-
text=True,
|
|
325
|
-
capture_output=True,
|
|
326
|
-
check=False,
|
|
327
|
-
)
|
|
328
|
-
(run_root / "code.diff").write_text(code_diff.stdout)
|
|
329
|
-
lines.append("## Code diff")
|
|
330
|
-
lines.append("")
|
|
331
|
-
if code_diff.stdout.strip():
|
|
332
|
-
lines.append("```diff")
|
|
333
|
-
lines.append(truncate(code_diff.stdout))
|
|
334
|
-
lines.append("```")
|
|
335
|
-
elif code_diff.returncode == 0:
|
|
336
|
-
lines.append("No difference between the two worktrees.")
|
|
337
|
-
else:
|
|
338
|
-
lines.append("Could not compute the worktree diff; see the artifacts.")
|
|
339
|
-
lines.append("")
|
|
340
|
-
|
|
341
|
-
lines.append("## Artifacts")
|
|
342
|
-
lines.append("")
|
|
343
|
-
lines.append(f"- Run directory: `{run_root}`")
|
|
344
|
-
lines.append(f"- model-a output: `{run_root / 'artifacts' / 'model-a' / 'output.txt'}`")
|
|
345
|
-
lines.append(f"- model-a changes: `{run_root / 'artifacts' / 'model-a' / 'changes.diff'}`")
|
|
346
|
-
lines.append(f"- model-b output: `{run_root / 'artifacts' / 'model-b' / 'output.txt'}`")
|
|
347
|
-
lines.append(f"- model-b changes: `{run_root / 'artifacts' / 'model-b' / 'changes.diff'}`")
|
|
348
|
-
lines.append(f"- Full output diff: `{run_root / 'output.diff'}`")
|
|
349
|
-
lines.append(f"- Full code diff: `{run_root / 'code.diff'}`")
|
|
350
|
-
lines.append("")
|
|
351
|
-
|
|
352
|
-
report_path = run_root / "report.md"
|
|
353
|
-
report_path.write_text("\n".join(lines) + "\n")
|
|
354
|
-
return report_path
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
def main() -> int:
|
|
358
|
-
parser = argparse.ArgumentParser(description="Compare two models on the same prompt in isolated worktrees.")
|
|
359
|
-
parser.add_argument("--model-a", required=True, help="Model for the first worktree.")
|
|
360
|
-
parser.add_argument("--model-b", required=True, help="Model for the second worktree.")
|
|
361
|
-
parser.add_argument(
|
|
362
|
-
"--thinking-a",
|
|
363
|
-
choices=list(THINKING_LEVELS),
|
|
364
|
-
help="Thinking level for model-a. Overrides a :level suffix on the model name.",
|
|
365
|
-
)
|
|
366
|
-
parser.add_argument(
|
|
367
|
-
"--thinking-b",
|
|
368
|
-
choices=list(THINKING_LEVELS),
|
|
369
|
-
help="Thinking level for model-b. Overrides a :level suffix on the model name.",
|
|
370
|
-
)
|
|
371
|
-
parser.add_argument("--prompt", help="Prompt to run in both worktrees.")
|
|
372
|
-
parser.add_argument("--prompt-file", help="Read the prompt from this file instead.")
|
|
373
|
-
parser.add_argument("--base", default="HEAD", help="Git ref to branch both worktrees from (default: HEAD).")
|
|
374
|
-
parser.add_argument("--timeout", type=int, default=1800, help="Per-model timeout in seconds (default: 1800).")
|
|
375
|
-
parser.add_argument("--keep", action="store_true", help="Keep the worktrees after the run.")
|
|
376
|
-
parser.add_argument("--dry-run", action="store_true", help="Create/tear down worktrees without invoking pi.")
|
|
377
|
-
parser.add_argument("--pi-bin", default=os.environ.get("PI_BIN", "pi"), help="pi executable (default: pi).")
|
|
378
|
-
args = parser.parse_args()
|
|
379
|
-
|
|
380
|
-
if args.prompt_file:
|
|
381
|
-
args.prompt = Path(args.prompt_file).read_text()
|
|
382
|
-
if not args.prompt or not args.prompt.strip():
|
|
383
|
-
parser.error("provide --prompt or --prompt-file")
|
|
384
|
-
if args.timeout <= 0:
|
|
385
|
-
parser.error("--timeout must be positive")
|
|
386
|
-
|
|
387
|
-
pi_bin = shutil.which(args.pi_bin) or (args.pi_bin if Path(args.pi_bin).exists() else None)
|
|
388
|
-
if not pi_bin:
|
|
389
|
-
print(f"error: pi executable not found: {args.pi_bin}", file=sys.stderr)
|
|
390
|
-
return 2
|
|
391
|
-
args.pi_bin = pi_bin
|
|
392
|
-
|
|
393
|
-
try:
|
|
394
|
-
repo_root = Path(git_text(["rev-parse", "--show-toplevel"], Path.cwd()))
|
|
395
|
-
except subprocess.CalledProcessError:
|
|
396
|
-
print("error: run sassy-compare from inside a git repository", file=sys.stderr)
|
|
397
|
-
return 2
|
|
398
|
-
|
|
399
|
-
try:
|
|
400
|
-
args.base_sha = git_text(["rev-parse", "--verify", f"{args.base}^{{commit}}"], repo_root)
|
|
401
|
-
except subprocess.CalledProcessError:
|
|
402
|
-
print(f"error: base ref not found: {args.base}", file=sys.stderr)
|
|
403
|
-
return 2
|
|
404
|
-
|
|
405
|
-
dirty = git_text(["status", "--porcelain"], repo_root)
|
|
406
|
-
if dirty:
|
|
407
|
-
print(
|
|
408
|
-
"warning: the repository has uncommitted changes; both worktrees use "
|
|
409
|
-
f"{args.base} and will not include them",
|
|
410
|
-
file=sys.stderr,
|
|
411
|
-
)
|
|
412
|
-
|
|
413
|
-
run_id = time.strftime("%Y%m%dT%H%M%SZ", time.gmtime()) + f"-{os.getpid()}"
|
|
414
|
-
compare_root = Path(os.environ.get("SASSY_COMPARE_DIR", Path.home() / ".pi" / "compare"))
|
|
415
|
-
run_root = compare_root / run_id
|
|
416
|
-
worktrees = run_root / "worktrees"
|
|
417
|
-
artifacts = run_root / "artifacts"
|
|
418
|
-
worktree_a = worktrees / "model-a"
|
|
419
|
-
worktree_b = worktrees / "model-b"
|
|
420
|
-
|
|
421
|
-
created: list[Path] = []
|
|
422
|
-
try:
|
|
423
|
-
for label in ("model-a", "model-b"):
|
|
424
|
-
out_dir = artifacts / label
|
|
425
|
-
out_dir.mkdir(parents=True, exist_ok=True)
|
|
426
|
-
worktrees.mkdir(parents=True, exist_ok=True)
|
|
427
|
-
|
|
428
|
-
for worktree in (worktree_a, worktree_b):
|
|
429
|
-
result = git(["worktree", "add", "--detach", str(worktree), args.base_sha], cwd=repo_root, check=False)
|
|
430
|
-
if result.returncode != 0:
|
|
431
|
-
raise RuntimeError(result.stderr.strip() or f"git worktree add failed for {worktree}")
|
|
432
|
-
created.append(worktree)
|
|
433
|
-
|
|
434
|
-
if args.dry_run:
|
|
435
|
-
summary_a = {"wall_ms": 0, "exit_code": 0, "timed_out": False, "turns": 0, "model_ms": 0,
|
|
436
|
-
"tool_ms": 0, "output_tokens": 0, "cost": 0.0, "errors": 0, "retries": 0,
|
|
437
|
-
"aborted": False, "stderr_tail": "", "thinking": args.thinking_a}
|
|
438
|
-
summary_b = {**summary_a, "thinking": args.thinking_b}
|
|
439
|
-
(artifacts / "model-a" / "output.txt").write_text("")
|
|
440
|
-
(artifacts / "model-b" / "output.txt").write_text("")
|
|
441
|
-
(artifacts / "model-a" / "files.txt").write_text("")
|
|
442
|
-
(artifacts / "model-b" / "files.txt").write_text("")
|
|
443
|
-
else:
|
|
444
|
-
summary_a = run_model(
|
|
445
|
-
"model-a", args.model_a, args.thinking_a, worktree_a, artifacts / "model-a", args.prompt, args
|
|
446
|
-
)
|
|
447
|
-
summary_b = run_model(
|
|
448
|
-
"model-b", args.model_b, args.thinking_b, worktree_b, artifacts / "model-b", args.prompt, args
|
|
449
|
-
)
|
|
450
|
-
|
|
451
|
-
report = write_report(
|
|
452
|
-
run_root,
|
|
453
|
-
args.model_a,
|
|
454
|
-
args.model_b,
|
|
455
|
-
args.thinking_a or thinking_from_model(args.model_a),
|
|
456
|
-
args.thinking_b or thinking_from_model(args.model_b),
|
|
457
|
-
args.prompt,
|
|
458
|
-
args.base,
|
|
459
|
-
worktree_a,
|
|
460
|
-
worktree_b,
|
|
461
|
-
summary_a,
|
|
462
|
-
summary_b,
|
|
463
|
-
args.dry_run,
|
|
464
|
-
)
|
|
465
|
-
|
|
466
|
-
if args.dry_run:
|
|
467
|
-
print("dry run complete; no models invoked")
|
|
468
|
-
else:
|
|
469
|
-
print(
|
|
470
|
-
f"model-a {args.model_a}: {fmt_ms(summary_a['wall_ms'])}, "
|
|
471
|
-
f"{summary_a['turns']} turns, {summary_a['errors']} error turns"
|
|
472
|
-
)
|
|
473
|
-
print(
|
|
474
|
-
f"model-b {args.model_b}: {fmt_ms(summary_b['wall_ms'])}, "
|
|
475
|
-
f"{summary_b['turns']} turns, {summary_b['errors']} error turns"
|
|
476
|
-
)
|
|
477
|
-
print(f"report: {report}")
|
|
478
|
-
if args.keep:
|
|
479
|
-
print(f"worktrees kept: {worktree_a} {worktree_b}")
|
|
480
|
-
return 0
|
|
481
|
-
finally:
|
|
482
|
-
if not args.keep:
|
|
483
|
-
for worktree in created:
|
|
484
|
-
git(["worktree", "remove", "--force", str(worktree)], cwd=repo_root, check=False)
|
|
485
|
-
git(["worktree", "prune"], cwd=repo_root, check=False)
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
if __name__ == "__main__":
|
|
489
|
-
try:
|
|
490
|
-
raise SystemExit(main())
|
|
491
|
-
except KeyboardInterrupt:
|
|
492
|
-
print("interrupted", file=sys.stderr)
|
|
493
|
-
raise SystemExit(130)
|