claude-dev-env 8.0.0 → 8.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/agents/clean-coder.md +4 -4
- package/.agents/agents/code-quality-agent.md +4 -4
- package/.agents/agents/plan-packet-validator.md +2 -2
- package/.agents/agents/session-advisor.md +1 -1
- package/.agents/skills/cert-classification-rule/SKILL.md +3 -3
- package/.agents/skills/e-code-review/SKILL.md +2 -2
- package/.agents/skills/e-code-review/reference/fix.md +1 -1
- package/.agents/skills/e-code-review/reference/loop.md +1 -1
- package/.agents/skills/e-code-review/reference/medium.md +4 -4
- package/.agents/skills/e-code-review/reference/xhigh.md +5 -5
- package/.agents/skills/everything-search/SKILL.md +3 -3
- package/.agents/skills/issue-tracker/SKILL.md +1 -1
- package/.agents/skills/orchestrator/SKILL.md +3 -3
- package/.agents/skills/orchestrator/reference/consult-the-orchestrator.md +1 -1
- package/.agents/skills/orchestrator/reference/executor-consult-block.md +1 -1
- package/.agents/skills/orchestrator-refresh/SKILL.md +2 -2
- package/.agents/skills/plugin-eval-standalone-skill/SKILL.md +96 -0
- package/.agents/skills/plugin-eval-standalone-skill/reference/graders-and-commands.md +108 -0
- package/.agents/skills/privacy-hygiene/SKILL.md +5 -6
- package/.agents/skills/privacy-hygiene/reference/sweep-procedure.md +5 -6
- package/.agents/skills/repairing-hook-boundaries/SKILL.md +1 -1
- package/.agents/skills/syncing-submodules/SKILL.md +1 -1
- package/.agents/skills/team-advisor/SKILL.md +9 -7
- package/.agents/skills/team-advisor/reference/advisor-docs-review.md +28 -1
- package/.agents/skills/usage-pause/SKILL.md +3 -3
- package/_shared/advisor/advisor-protocol.md +7 -3
- package/_shared/advisor/reference/advisor-block.md +8 -1
- package/audit-rubrics/audit-categories.json +0 -4
- package/audit-rubrics/category_rubrics/category-e-dead-code.md +1 -1
- package/audit-rubrics/category_rubrics/category-f-silent-failures.md +2 -2
- package/audit-rubrics/category_rubrics/category-i-concurrency.md +1 -1
- package/audit-rubrics/category_rubrics/category-j-code-rules-compliance.md +1 -2
- package/audit-rubrics/category_rubrics/category-m-producer-consumer-cardinality.md +1 -1
- package/audit-rubrics/category_rubrics/category-n-test-name-scenario-verifier.md +5 -5
- package/audit-rubrics/category_rubrics/category-o-docstring-vs-impl-drift.md +5 -5
- package/audit-rubrics/category_rubrics/category-p-name-vs-behavior-contract.md +3 -3
- package/audit-rubrics/category_rubrics/category-q-cross-surface-claims.md +1 -1
- package/audit-rubrics/prompts/category-a-api-contracts.md +2 -2
- package/audit-rubrics/prompts/category-b-selector-engine-compat.md +3 -3
- package/audit-rubrics/prompts/category-c-resource-cleanup.md +3 -3
- package/audit-rubrics/prompts/category-d-scoping-and-ordering.md +1 -1
- package/audit-rubrics/prompts/category-e-dead-code.md +5 -5
- package/audit-rubrics/prompts/category-f-silent-failures.md +12 -12
- package/audit-rubrics/prompts/category-g-bounds-and-overflow.md +2 -2
- package/audit-rubrics/prompts/category-h-security-boundaries.md +4 -4
- package/audit-rubrics/prompts/category-i-concurrency.md +6 -6
- package/audit-rubrics/prompts/category-j-code-rules-compliance.md +6 -29
- package/audit-rubrics/prompts/category-m-producer-consumer-cardinality.md +1 -1
- package/audit-rubrics/prompts/category-n-test-name-scenario-verifier.md +7 -7
- package/audit-rubrics/prompts/category-o-docstring-vs-impl-drift.md +2 -2
- package/audit-rubrics/prompts/category-p-name-vs-behavior-contract.md +3 -3
- package/audit-rubrics/prompts/category-q-cross-surface-claims.md +2 -2
- package/bin/ever-shipped-skills.mjs +1 -0
- package/bin/ever-shipped-skills.test.mjs +4 -0
- package/docs/BDD_SCENARIO_QUALITY.md +2 -2
- package/docs/CODE_RULES.md +7 -7
- package/docs/TEST_QUALITY.md +1 -1
- package/docs/nas-ssh-invocation.md +1 -1
- package/docs/references/advisor-tool.md +2 -2
- package/docs/references/prose-style-enforcement.md +1 -1
- package/docs/references/team-advisor-skill.md +2 -2
- package/docs/references/weak-executor-advisor.md +1 -1
- package/hooks/blocking/test_code_rules_enforcer_magic_allowlist.py +1 -1
- package/package.json +1 -1
- package/rules/anti-corollary-tests.md +3 -1
- package/rules/archiving-agent-config.md +67 -0
- package/rules/asd-ste100-language.md +1 -1
- package/rules/ci-owns-the-gate.md +15 -0
- package/rules/cleanup-temp-files.md +1 -1
- package/rules/code-standards.md +2 -2
- package/rules/docstring-prose-matches-implementation.md +1 -1
- package/rules/explore-thoroughly.md +2 -2
- package/rules/failure-blast-radius.md +9 -8
- package/rules/git-workflow.md +7 -4
- package/rules/long-horizon-autonomy.md +3 -1
- package/rules/nas-ssh-invocation.md +1 -1
- package/rules/research-mode.md +3 -1
- package/rules/retired-hook-prose.md +10 -2
- package/rules/testing.md +3 -3
- package/rules/verify-before-asking.md +3 -0
- package/rules/verify-runtime-state.md +2 -2
- package/scripts/active_capability_references.py +74 -19
- package/scripts/banned_prose_words.py +159 -0
- package/scripts/dev_env_scripts_constants/active_capability_constants.py +12 -10
- package/scripts/dev_env_scripts_constants/banned_prose_word_constants.py +95 -0
- package/scripts/policy_lint/adapter_retired_hook_prose.py +17 -7
- package/scripts/policy_lint/adapters.py +6 -6
- package/scripts/policy_lint/config/constants.py +8 -0
- package/scripts/policy_lint/registry.py +1 -1
- package/scripts/sync_to_cursor/rules.py +1 -1
- package/scripts/test_active_capability_references.py +100 -33
- package/scripts/tests/test_banned_prose_words.py +103 -0
- package/scripts/tests/test_codex_compat_materializer.py +3 -3
- package/scripts/tests/test_engine.py +1 -1
- package/scripts/tests/test_policy_lint_rules_retired_hook_prose.py +46 -0
- package/scripts/tests/test_rules.py +12 -1
- package/system-prompts/software-engineer.xml +7 -7
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: team-advisor
|
|
3
3
|
description: >-
|
|
4
|
-
|
|
5
|
-
it before substantive work, completion, commits, or when stuck.
|
|
6
|
-
|
|
7
|
-
'
|
|
8
|
-
|
|
4
|
+
Bind one advisor, the built-in advisor tool or a warm session-advisor, and
|
|
5
|
+
consult it before substantive work, completion, commits, or when stuck.
|
|
6
|
+
Triggers:
|
|
7
|
+
'team-advisor', 'team advisor', 'second opinion', 'consult the advisor',
|
|
8
|
+
'get an advisor', 'check my approach with an advisor'.
|
|
9
9
|
---
|
|
10
10
|
|
|
11
11
|
# Team Advisor
|
|
@@ -24,7 +24,7 @@ One warm advisor at the strongest tier this session can reach. This session is t
|
|
|
24
24
|
## Bind
|
|
25
25
|
|
|
26
26
|
1. Name the session identity first (protocol **Host profiles**), then walk the model floor.
|
|
27
|
-
2. Claude:
|
|
27
|
+
2. Claude: when `advisor` is in this session's tool list, the built-in advisor tool is the advisor. Spawn nothing and call `advisor()` at each consult point. Otherwise spawn Fable in-session at `ADVISOR_EFFORT` (default low). When Fable is out of usage, bind Astra at the same effort. Codex: Astra in-session. Third-party: headless Fable then Astra. When the host's walk fails, fail closed.
|
|
28
28
|
3. Name: `team-advisor-agent` on Claude (Agent spawn of `session-advisor`); a native Astra subagent on Codex; one CLI `session_id` on a third-party host via the protocol Claude-chain or Astra helper.
|
|
29
29
|
4. Skip the multi-consumer "who you are" opener. This session is the sole consumer.
|
|
30
30
|
5. When the bind or reply path fails, fail closed and report to the user. On a third-party host, only the bound advisor issues ENDORSE / CORRECTION / PLAN / STOP.
|
|
@@ -39,10 +39,12 @@ Follow **When to call**, **Hard rule**, and **How to treat advice** in `advisor-
|
|
|
39
39
|
|
|
40
40
|
Build every first brief with [`_shared/advisor/reference/consult-format.md`](../../_shared/advisor/reference/consult-format.md). Later briefs carry only the delta and changed evidence.
|
|
41
41
|
|
|
42
|
+
With the built-in advisor tool, call `advisor()` with no brief. The tool forwards the whole transcript. Its reply is free text, not one of the four signal words, so weigh it per **How to treat advice** in `advisor-tool.md`.
|
|
43
|
+
|
|
42
44
|
Aim for two consults on a normal task: one after orientation and one after writes and validation. Reserve a third for advisory recovery or reconciliation guidance, and add a consult when a material fork produces new evidence. This is an advisory target owned by the task, not a cap or gate.
|
|
43
45
|
|
|
44
46
|
## Constraints
|
|
45
47
|
|
|
46
|
-
- One bind per session; this session owns spawn, in-session Astra spawn, or CLI bind, drift re-bind, and shutdown.
|
|
48
|
+
- One bind per session; this session owns the built-in tool choice, spawn, in-session Astra spawn, or CLI bind, drift re-bind, and shutdown.
|
|
47
49
|
- Bind at or above the protocol floor for this host.
|
|
48
50
|
- The advisor only answers (messaging); the session runs tools and posts.
|
|
@@ -125,7 +125,7 @@ directly; the wording lives under **Brevity cue** in
|
|
|
125
125
|
Direct address works because the advisor reads the executor's prompt as
|
|
126
126
|
quoted context: "instructions that address the advisor directly are followed
|
|
127
127
|
much more reliably than third-person descriptions." A brief that asks for
|
|
128
|
-
roughly 80 percent of the
|
|
128
|
+
roughly 80 percent of the ceiling raises consult frequency while
|
|
129
129
|
lowering total cost.
|
|
130
130
|
|
|
131
131
|
Typical advisor output runs 400–700 text tokens, or 1,400–1,800 with
|
|
@@ -205,3 +205,30 @@ successful task.
|
|
|
205
205
|
|
|
206
206
|
Track, per route: completion rate, regression rate, tool calls, tokens by
|
|
207
207
|
tier, and latency.
|
|
208
|
+
|
|
209
|
+
## I. Verifying the built-in tool path
|
|
210
|
+
|
|
211
|
+
Repository practice, observed on Claude Code 2.1.277: `claude plugin eval`
|
|
212
|
+
cannot reach the built-in tool path. Its sandbox turns off feature-flag
|
|
213
|
+
fetching, and `/advisor` replies "isn't available in this environment". A
|
|
214
|
+
plugin-eval suite therefore grades the warm-agent path alone.
|
|
215
|
+
|
|
216
|
+
Drive the built-in tool path with a headless run, once with the skill and once
|
|
217
|
+
without:
|
|
218
|
+
|
|
219
|
+
```text
|
|
220
|
+
claude -p "<task>" --advisor opus --plugin-dir <wrapper> \
|
|
221
|
+
--setting-sources project --strict-mcp-config \
|
|
222
|
+
--allowedTools "Read,Glob,Grep,Skill,Agent,SendMessage,Write,Edit" \
|
|
223
|
+
--permission-mode dontAsk --output-format stream-json --verbose
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Count main-thread content blocks with `type: server_tool_use` and
|
|
227
|
+
`name: advisor`. Keep each run's folder outside `<wrapper>`, because
|
|
228
|
+
`dontAsk` denies writes under a loaded plugin directory. Read
|
|
229
|
+
`permission_denials` in each run before you compare the two arms.
|
|
230
|
+
|
|
231
|
+
Run a positive control first, with a prompt that asks for the advisor by name.
|
|
232
|
+
A zero count means the tool was never attached, or the model chose not to
|
|
233
|
+
call it. The control tells the two apart. An empty `CLAUDE_CONFIG_DIR` drops
|
|
234
|
+
the login, so keep the user config and use `--setting-sources project`.
|
|
@@ -33,7 +33,7 @@ On exit 0 the script prints one JSON object:
|
|
|
33
33
|
| `session_utilization` | Percent of the 5-hour window spent (null on override) |
|
|
34
34
|
| `weekly_utilization` | Percent of the weekly window spent (null on override) |
|
|
35
35
|
| `weekly_resets_at` | When the weekly window resets (null on override) |
|
|
36
|
-
| `weekly_near_cap` |
|
|
36
|
+
| `weekly_near_cap` | `true` when the weekly meter is at or past the warn threshold |
|
|
37
37
|
|
|
38
38
|
On exit 2 the script prints `{"error": ...}`. Ask the user for a manual reset time via AskUserQuestion (offer clock-time and duration examples), then rerun with `--override`.
|
|
39
39
|
|
|
@@ -68,7 +68,7 @@ Before the first sleep:
|
|
|
68
68
|
|
|
69
69
|
The stage plan (`stages_seconds` from the resolver) keeps every stage under the maximum stage length and ends with a short tail so the final firing lands just past the reset. Stage sizing constants live in `scripts/usage_pause_constants/resolve_usage_window_constants.py`; a leftover too short to stand alone folds into the tail.
|
|
70
70
|
|
|
71
|
-
Every wakeup does exactly three things — ping, record, schedule — and dispatches no new work. The cache rationale: an unpinged idle agent left past the cache window resumes cold and re-reads its whole transcript, so pings run tighter than the cache lifetime and keep each agent's next
|
|
71
|
+
Every wakeup does exactly three things — ping, record, schedule — and dispatches no new work. The cache rationale: an unpinged idle agent left past the cache window resumes cold and re-reads its whole transcript, so pings run tighter than the cache lifetime and keep each agent's next invocation cheap.
|
|
72
72
|
|
|
73
73
|
### Stage wakeup prompt template
|
|
74
74
|
|
|
@@ -78,7 +78,7 @@ Remaining chain (seconds): <remaining_stage_durations>. Cancelled crons: <cancel
|
|
|
78
78
|
1. Ping every live agent AND every idle-but-warm agent with exactly this one line:
|
|
79
79
|
"status checkpoint: reply with where you stand, do not start new work"
|
|
80
80
|
An unpinged idle agent past the cache window resumes cold and re-reads its whole
|
|
81
|
-
transcript; the ping keeps its next
|
|
81
|
+
transcript; the ping keeps its next invocation cheap.
|
|
82
82
|
2. Record any finished agent results on the task list. Dispatch nothing.
|
|
83
83
|
3. Take the first duration off the remaining chain and ScheduleWakeup it, passing this
|
|
84
84
|
same prompt with that duration removed and the stage number advanced. When the
|
|
@@ -62,7 +62,9 @@ On Codex: Astra is the in-session default. The `ADVISOR_ASTRA` flag is not requi
|
|
|
62
62
|
|
|
63
63
|
### Claude host
|
|
64
64
|
|
|
65
|
-
|
|
65
|
+
When `advisor` is in the session's tool list, the built-in advisor tool is the advisor. The user turns it on with `/advisor`, the `advisorModel` setting, or `--advisor`. Spawn nothing and call `advisor()` at each consult point. The tool forwards the whole transcript, so a consult needs no packet. Its reply is free text, so the four signal words do not apply; weigh it per `docs/references/advisor-tool.md` **How to treat advice**. It is a server tool, so hooks and the spawn-walk log do not see it. A Fable or Opus 5 advisor returns an encrypted result that a log cannot read. When a readable record matters, use the warm agent below.
|
|
66
|
+
|
|
67
|
+
When `advisor` is not in the tool list, use the **Model floor** ladder below (Fable first, then Astra when Fable is out of usage).
|
|
66
68
|
Warm-up spawns `subagent_type: session-advisor` via the Agent tool for Fable; consults go through `SendMessage` to that warm agent. Astra uses the Codex helper.
|
|
67
69
|
Assemble and paste each executor's Advisor block per the **Advisor block** section.
|
|
68
70
|
|
|
@@ -85,6 +87,7 @@ Executors report to the orchestrating session; that session consults the bound a
|
|
|
85
87
|
|
|
86
88
|
## Model floor
|
|
87
89
|
|
|
90
|
+
The built-in advisor tool sits outside this ladder. Claude Code and the API accept it only when the advisor is at least as capable as the main model, and that check is its floor.
|
|
88
91
|
On Claude and ThirdParty the advisor ladder is `Fable` first, then Astra (flag-gated, Codex CLI) when Fable is out of usage.
|
|
89
92
|
On Codex the walk is Astra only, in-session.
|
|
90
93
|
Opus is not an advisor candidate. `Sonnet` and `Haiku` are executor tiers only.
|
|
@@ -103,7 +106,8 @@ The validator checks ladder shape only; host policy sits on top.
|
|
|
103
106
|
|
|
104
107
|
## Warm-up (once per session)
|
|
105
108
|
|
|
106
|
-
On a **Claude host
|
|
109
|
+
On a **Claude host** with `advisor` in the tool list, skip the warm-up; the built-in tool needs none.
|
|
110
|
+
On a **Claude host** without it, spawn `session-advisor` at the Fable alias with the charter as its prompt. When Fable is out of usage and the Astra rung is open, bind Astra through the Codex helper instead.
|
|
107
111
|
Full spawn fields and the charter template: [`reference/warm-up.md`](reference/warm-up.md).
|
|
108
112
|
|
|
109
113
|
On a **Codex host**, spawn a native in-session Astra subagent with the same charter. Bind fields: [`reference/identity.md`](reference/identity.md) and [`reference/warm-up.md`](reference/warm-up.md).
|
|
@@ -141,7 +145,7 @@ Modes and failover, the tier-to-alias table, brief piping, and `--resume` sessio
|
|
|
141
145
|
|
|
142
146
|
**Codex host:** do not use this runner as the primary path. Astra binds in-session.
|
|
143
147
|
|
|
144
|
-
**Claude host:** fall back to this runner exactly when one of these holds:
|
|
148
|
+
**Claude host:** the built-in advisor tool never falls back to this runner. On the warm-agent path, fall back to this runner exactly when one of these holds:
|
|
145
149
|
- The Agent-tool spawn errors at every candidate tier down to the floor. The tool itself is unavailable.
|
|
146
150
|
- `SendMessage` to the shared advisor errors, or draws no reply within the bound in `ADVISOR_SENDMESSAGE_REPLY_WAIT_SECONDS` (120) in `$HOME/.claude/_shared/advisor/scripts/config/advisor_scripts_constants/model_tier_run_validator_constants.py`, and a re-spawn also fails.
|
|
147
151
|
- The running session is itself a subagent barred from spawning further agents.
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
Detail behind the **Advisor block** section of [`advisor-protocol.md`](../advisor-protocol.md).
|
|
4
4
|
Open this when assembling the block for an executor spawn prompt.
|
|
5
5
|
|
|
6
|
-
Assembly order: one transport preamble picked by host profile, then the shared core, then the weak-executor add-on for an executor at Sonnet or below.
|
|
6
|
+
Assembly order: one transport preamble picked by host profile, then the shared core, then the weak-executor add-on for an executor at Sonnet or below. A Claude host with the built-in advisor tool takes its own block and skips the preamble and the shared core.
|
|
7
7
|
Paste the assembled block at the **top** of the spawn prompt, ahead of any other sentence that mentions the advisor.
|
|
8
8
|
The assembled block is self-contained. The executor receives this text alone, so every path it names is absolute.
|
|
9
9
|
The consult rules it restates are owned by [`consult-format.md`](consult-format.md); edit that file first and carry the change here.
|
|
@@ -12,6 +12,13 @@ The consult rules it restates are owned by [`consult-format.md`](consult-format.
|
|
|
12
12
|
|
|
13
13
|
> A shared session advisor named `<name>` is reachable via SendMessage; send each consult to it directly by that name.
|
|
14
14
|
|
|
15
|
+
## Block for a Claude host with the built-in advisor tool
|
|
16
|
+
|
|
17
|
+
Subagents inherit the session's advisor tool, so the executor calls it directly. Paste this block alone, then the weak-executor add-on for an executor at Sonnet or below. The shared core's packet and signal-word lines do not apply, and neither does the add-on's 80-word line.
|
|
18
|
+
|
|
19
|
+
> Your session has the built-in `advisor` tool. Call `advisor()` before locking in a nontrivial approach, once you believe your assignment is done, before any hard-to-reverse action, when the same failure repeats or progress has stalled, and when the chosen approach is being reconsidered.
|
|
20
|
+
> The tool reads your whole transcript, so call it with no brief. Its reply is free text. Give it serious weight; when your evidence contradicts it, call again and name the conflict.
|
|
21
|
+
|
|
15
22
|
## Transport preamble for a Codex host
|
|
16
23
|
|
|
17
24
|
> A shared session advisor named `<name>` is reachable as a native Codex Astra subagent; send each consult to it in-session by that name.
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
| E6 | Removed-but-not-deleted symbol references | Symbols renamed/removed elsewhere with stale import or call sites left behind. |
|
|
27
27
|
| E7 | Test fixtures / helpers defined but never used | Pytest fixtures, test data builders, mock factories with no callers. |
|
|
28
28
|
| E8 | Stub / placeholder code without tracking | Judge `pass`, `...`, and `raise NotImplementedError` directly; track follow-up work outside source code. |
|
|
29
|
-
| E9 | Constants-module exports with no importer | A module-level `UPPER_SNAKE` constant added to a `*_constants.py` / `config/` module that no module in the repo imports and that the constants file itself never references.
|
|
29
|
+
| E9 | Constants-module exports with no importer | A module-level `UPPER_SNAKE` constant added to a `*_constants.py` / `config/` module that no module in the repo imports and that the constants file itself never references. No write-time check reads a constants module, and every name such a module exports carries zero in-file references by design, so a dead export reaches the audit untouched. Distinguish dead from live by grepping the whole repo for each constant name: a sibling such as `MEDIUM_TERMINAL` imported by a consumer module is live; a `MEDIUM_TEXT` that no `from ... import` line and no in-file reference names is dead (CODE_RULES 9.8). Remove the dead export. |
|
|
30
30
|
|
|
31
31
|
---
|
|
32
32
|
|
|
@@ -20,14 +20,14 @@
|
|
|
20
20
|
|---|---|---|
|
|
21
21
|
| F1 | Catch-all except clauses | `except:` (bare), `except Exception:`, `except BaseException:` followed by `pass` / `continue` / log-only. |
|
|
22
22
|
| F2 | Errors logged then swallowed | `logger.error(...)` followed by `return None` / `return default` without re-raise. |
|
|
23
|
-
| F3 | Default fallback values masking failure | `dict.get(key, default)` where the absence of the key is itself a bug; `or default` short-circuits hiding `None`. Includes the stale-payload-key shape: a `payload.get("KEY", "")` (or `payload["KEY"]` wrapped in a fallback) read against an external-input dict whose contract the diff migrated — the rest of the module reads the payload through a different key set named in the docstring and bound to same-named variables, while this lone read targets a dropped key, so it resolves to the default on every
|
|
23
|
+
| F3 | Default fallback values masking failure | `dict.get(key, default)` where the absence of the key is itself a bug; `or default` short-circuits hiding `None`. Includes the stale-payload-key shape: a `payload.get("KEY", "")` (or `payload["KEY"]` wrapped in a fallback) read against an external-input dict whose contract the diff migrated — the rest of the module reads the payload through a different key set named in the docstring and bound to same-named variables, while this lone read targets a dropped key, so it resolves to the default on every payload and silently records an empty value into the field it feeds. The audit teammate lists every string-literal key read from each `*_payload` / event / request dict, checks each key against the payload contract the module's docstring and other reads set up, and flags a key read at one site that no docstring, second read, or same-named binding anchors. A key consumed inline with a meaningful default (`resolve(payload.get("cwd", "."))`) is legitimate, not stale — the flag is the dropped key whose value the field needs but never receives. |
|
|
24
24
|
| F4 | Async task error swallowing | `asyncio.create_task(...)` without exception observation; `gather(..., return_exceptions=True)` consumed loosely. |
|
|
25
25
|
| F5 | Boolean / status returns identical on success and failure | A function returns `True` on the happy path and `True` on the catch-all error path. |
|
|
26
26
|
| F6 | Ignored return values from fallible calls | `subprocess.run` without `check=True` and unchecked `returncode`; `os.write` return value discarded. |
|
|
27
27
|
| F7 | PowerShell error-suppression patterns | `-ErrorAction SilentlyContinue` followed by `.Property` access; `2>$null` or `*>$null`; `$?` not consulted. |
|
|
28
28
|
| F8 | Test-level swallowing | Tests that catch and log instead of asserting; `pytest.warns` used instead of `pytest.raises`. |
|
|
29
29
|
| F9 | Gate-validator self-defeat via parse-failure swallow | A new gate / validator / hook check parses input as code (`ast.parse`) and catches the parse error with `return []` / `return None` / `return True` (clean signal). When the dispatcher feeds the check a partial fragment (e.g., the Edit tool's `new_string` rather than the full file content), the parse always fails and the check silently fires zero findings. The audit teammate lists every new `ast.parse` / `tokenize` / `json.loads` / `yaml.safe_load` call in gate code and asks: does the catch branch produce a clean signal that would mask the gate being broken? |
|
|
30
|
-
| F10 | Guard helper returns success-default on unverifiable input | A guard helper / predicate / classifier short-circuits to `True` / `Ok` / the success sentinel when it cannot validate the input shape (missing positional args, wrong arity, unexpected None). Guards must default-deny when they cannot verify; default-allow on unverifiable input masks
|
|
30
|
+
| F10 | Guard helper returns success-default on unverifiable input | A guard helper / predicate / classifier short-circuits to `True` / `Ok` / the success sentinel when it cannot validate the input shape (missing positional args, wrong arity, unexpected None). Guards must default-deny when they cannot verify; default-allow on unverifiable input masks violations downstream. |
|
|
31
31
|
|
|
32
32
|
---
|
|
33
33
|
|
|
@@ -33,6 +33,6 @@
|
|
|
33
33
|
The reusable Variant C template for Category I is in [`../prompts/category-i-concurrency.md`](../prompts/category-i-concurrency.md). Inline your artifact under `## Source material` and adapt the sub-bucket bullets to your project's concurrency model.
|
|
34
34
|
|
|
35
35
|
For a literal worked example using PR #394, see [`category-a-api-contracts.md`](category-a-api-contracts.md). Category I walks for that diff:
|
|
36
|
-
- I4: TOCTOU between `os.walk` enumerating a directory and `os.path.getctime` / `os.rmdir` on the same path — another process could delete or repopulate the dir in the window. The `try/except OSError` handles the race correctly (Category F notes the same blocks for silent-failure concerns; here they're
|
|
36
|
+
- I4: TOCTOU between `os.walk` enumerating a directory and `os.path.getctime` / `os.rmdir` on the same path — another process could delete or repopulate the dir in the window. The `try/except OSError` handles the race correctly (Category F notes the same blocks for silent-failure concerns; here they're protective).
|
|
37
37
|
- I4 (PowerShell): `Test-Path $Target` followed by `Register-ScheduledTask` — directory could be deleted between the check and the registration. Low-impact since the schedule still registers.
|
|
38
38
|
- I1, I2, I3, I5–I8: not applicable — script is single-threaded synchronous Python with no asyncio, no shared mutable state across processes.
|
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
**Examples of Category J findings:**
|
|
6
6
|
- A literal `60` appears in a production function body (magic value rule).
|
|
7
7
|
- A new `MAX_RETRIES = 3` declared at module scope outside `config/`.
|
|
8
|
-
- A parameter named `ctx`
|
|
8
|
+
- A parameter named `ctx` where the name `context` belongs (abbreviation rule).
|
|
9
9
|
- A function that returns a value with no return-type annotation.
|
|
10
10
|
- A new `# explains the loop logic` comment added to production code.
|
|
11
11
|
|
|
@@ -20,7 +20,6 @@
|
|
|
20
20
|
| J1 | Magic values in production function bodies | Literals other than `0`, `1`, `-1` inside production function bodies. Test files exempt. |
|
|
21
21
|
| J2 | String-template magic | f-strings whose structural literal text (paths, URLs, patterns) belongs in `config/`. |
|
|
22
22
|
| J3 | Constants location | Module-level `UPPER_SNAKE = ...` outside `config/` in production code. Exempt path families: `config/*`, `/migrations/`, `/workflow/`, `_tab.py`, `/states.py`, `/modules.py`, test files. |
|
|
23
|
-
| J4 | File-global use-count | A file-global constant referenced by fewer than two methods/functions/classes in the same file. |
|
|
24
23
|
| J5 | Abbreviations | `ctx`, `cfg`, `msg`, `btn`, `idx`, `cnt`, `elem`, `val`, `tmp`, `str`, `num`, `arr`, `obj`, `fn`, `cb`, `req`, `res`. (Loop counters `i`/`j`/`k` and `e` for exceptions are exempt.) |
|
|
25
24
|
| J6 | Vague names | `result`, `data`, `output`, `response`, `value`, `item`, `temp`, `info`, `stuff`, `thing`. Vague prefixes: `handle`, `process`, `manage`, `do`. |
|
|
26
25
|
| J7 | Type hints | Missing type annotation on a parameter or return; presence of `Any` or `# type: ignore`. |
|
|
@@ -41,4 +41,4 @@ The reusable Variant C template for Category M is in [`../prompts/category-m-pro
|
|
|
41
41
|
|
|
42
42
|
Categories A–K each examine one side of an interface in isolation. Category M examines the cardinality contract spanning two sides: the producer's "can my return value contain duplicates?" question and the consumer's "do I tolerate duplicates?" answer. A reviewer walking only A–K reads the producer, finds it correct on its own terms, and approves it — then reads the consumer separately and finds it also correct on its own terms. The bug emerges only when the two are exercised against the same input together.
|
|
43
43
|
|
|
44
|
-
The pa#143 F10 case is the canonical worked example: `_extract_paths_from_everything_cli_stdout` returned `list[Path]` by walking the `es.exe` stdout line-by-line. The consumer iterated the list directly and ran one `INSERT INTO watched_dirs(path) VALUES (...)` per element against a table with a `UNIQUE(path)` constraint. When the subprocess emitted the same path on two lines (
|
|
44
|
+
The pa#143 F10 case is the canonical worked example: `_extract_paths_from_everything_cli_stdout` returned `list[Path]` by walking the `es.exe` stdout line-by-line. The consumer iterated the list directly and ran one `INSERT INTO watched_dirs(path) VALUES (...)` per element against a table with a `UNIQUE(path)` constraint. When the subprocess emitted the same path on two lines (an edge case — Everything's stdout can repeat results across drive letters), the list carried the path twice, so the writeback ran the same `INSERT` twice; the second `INSERT` raised `sqlite3.IntegrityError: UNIQUE constraint failed: watched_dirs.path` and the entire watchdog crashed. The fix was to change the producer to `frozenset[Path]`, which deduplicates at the boundary so each path reaches the writeback once; the consumer's per-element `INSERT` loop was already correct. The contract was the bug.
|
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
# Category N — Test-name scenario verifier
|
|
2
2
|
|
|
3
|
-
**What this category audits:** tests whose names assert a specific scenario (`test_*_when_*`, `test_*_at_*`, `test_*_under_*`, `test_*_with_*`) must demonstrate via fixture inspection, monkeypatch state, or environment setup that the named condition is in effect when the system under test runs. Common failure: a test named `test_resolves_under_unc_prefix` constructs a regular drive-letter path, asserts the function returns a non-empty result, and passes — but the UNC-prefix branch was never exercised. The test is a load-bearing witness that the named scenario works, but it
|
|
3
|
+
**What this category audits:** tests whose names assert a specific scenario (`test_*_when_*`, `test_*_at_*`, `test_*_under_*`, `test_*_with_*`) must demonstrate via fixture inspection, monkeypatch state, or environment setup that the named condition is in effect when the system under test runs. Common failure: a test named `test_resolves_under_unc_prefix` constructs a regular drive-letter path, asserts the function returns a non-empty result, and passes — but the UNC-prefix branch was never exercised. The test is a load-bearing witness that the named scenario works, but it only exercises the no-op default path.
|
|
4
4
|
|
|
5
|
-
**Why this category is its own bucket:** Categories A–K catch defects in the production code. Category N catches defects in the *test corpus's claim to coverage*. A green test suite is only meaningful when each test's name accurately describes what was exercised. When a test passes "for the wrong reason" — e.g., the assertion matches a substring that appears in both the scenario-named branch AND the default branch — the suite reports coverage of a scenario it never
|
|
5
|
+
**Why this category is its own bucket:** Categories A–K catch defects in the production code. Category N catches defects in the *test corpus's claim to coverage*. A green test suite is only meaningful when each test's name accurately describes what was exercised. When a test passes "for the wrong reason" — e.g., the assertion matches a substring that appears in both the scenario-named branch AND the default branch — the suite reports coverage of a scenario it never ran.
|
|
6
6
|
|
|
7
7
|
**Examples of Category N findings:**
|
|
8
8
|
- Cross-platform tests named `test_*_on_windows` / `test_*_on_linux` that don't gate on `sys.platform` or `monkeypatch.setattr(os, "name", ...)` and run unchanged on every platform. (ccc#476 F5, F21, F23, F26, F27)
|
|
9
9
|
- String-shape tests that exercise only the no-op branch (`assert result == ""` after constructing an input that hits the early-return path, not the named scenario). (pa#135 F11, F15)
|
|
10
10
|
- Integration tests with assertions like `<substring> not in executed_sql` where the substring shape never matches the SQL fragment shape — the assertion cannot fail by construction. (pa#136 F50)
|
|
11
11
|
- Path-decision tests for `is_test_file` / `is_hook_infrastructure` / `_resolve_*_path` without a parametric matrix of canonical edge cases (empty string, tilde-prefix, UNC, drive-letter, symlinked, `..`-containing, trailing slash).
|
|
12
|
-
- A test resolves `Path(__file__).parents[3]` expecting the `claude-dev-env/` package root, but the parents chain
|
|
12
|
+
- A test resolves `Path(__file__).parents[3]` expecting the `claude-dev-env/` package root, but the parents chain stops at `skills/` — the test cannot fail for the right reason and the asserted directory wiring is broken by construction.
|
|
13
13
|
- A test imports the same function twice under two names (`from path_utils import is_config_file` plus `from path_utils import is_config_file as path_utils_is_config_file`) and asserts the two bound names produce the same result — the assertion cannot fail because both names are the same function object; the appearance of two parallel implementations is fake.
|
|
14
14
|
|
|
15
15
|
**Companion reference:** see `../source-material-section-types.md`.
|
|
@@ -26,12 +26,12 @@ Decomposition is by the **kind of scenario claim** the test name makes vs the ev
|
|
|
26
26
|
| N2 | Path-decision parametric matrices | Tests for `is_*_path` / `_resolve_*_path` / `*_path_exemptions` modules ship a parametrized matrix covering: empty string, single filename, tilde-prefix, UNC path, drive-letter path, symlinked path, `..`-containing path, trailing-slash path |
|
|
27
27
|
| N3 | Tests that pass "for the wrong reason" | The assertion's substring matches both the scenario-named branch and the default branch — the test cannot distinguish which branch executed |
|
|
28
28
|
| N4 | No-op branch exercised by scenario name | The constructed input hits an early-return / guard / no-feature-installed branch BEFORE the scenario-named code runs; the assertion succeeds against the no-op output |
|
|
29
|
-
| N5 | Assertion shape mismatch | The assertion shape (`<substring> not in result`, `result == ""`, `len(result) > 0`) cannot fail by construction against the
|
|
29
|
+
| N5 | Assertion shape mismatch | The assertion shape (`<substring> not in result`, `result == ""`, `len(result) > 0`) cannot fail by construction against the return type / value space |
|
|
30
30
|
| N6 | Cross-platform scenario gating | Tests named `_on_<platform>` MUST gate on `sys.platform`, `monkeypatch.setattr(os, ...)`, or `@pytest.mark.skipif(sys.platform != "<platform>")` — bare scenario names that run unchanged across platforms claim more than they prove |
|
|
31
31
|
| N7 | Time / clock scenario gating | Tests named `_after_<duration>` / `_at_midnight` / `_during_business_hours` MUST inject a frozen clock (`freezegun.freeze_time`, `monkeypatch.setattr(time, "time", ...)`) — wall-clock tests are non-deterministic and may pass against the wrong scenario |
|
|
32
32
|
| N8 | Concurrent / load scenario gating | Tests named `_under_load` / `_with_concurrent_writers` MUST spawn the concurrent workers and `wait()` on them — single-threaded tests cannot claim concurrent-scenario coverage |
|
|
33
33
|
| N9 | Neutral-named tests (out of scope) | Tests named `test_returns_empty_list_for_unknown_key` / `test_handles_y` (no scenario claim in the name) are NOT subject to N1–N8; flag them only for assertion-shape mismatches under N5 |
|
|
34
|
-
| N10 | Test fixture wiring correctness | The test's fixture / path / import wiring resolves to the artifact the test name claims. Path arithmetic (`Path(__file__).parents[k]`) reaches the directory the assertion expects — verify by walking the parents chain symbolically. Same-symbol dual imports (`from m import f` plus `from m import f as f_alias`) bind two names to the same function object, so any parity assertion between them is true by construction. Fixture file lookups (`open(Path(__file__).parent / 'fixture.txt')`) reach a file that
|
|
34
|
+
| N10 | Test fixture wiring correctness | The test's fixture / path / import wiring resolves to the artifact the test name claims. Path arithmetic (`Path(__file__).parents[k]`) reaches the directory the assertion expects — verify by walking the parents chain symbolically. Same-symbol dual imports (`from m import f` plus `from m import f as f_alias`) bind two names to the same function object, so any parity assertion between them is true by construction. Fixture file lookups (`open(Path(__file__).parent / 'fixture.txt')`) reach a file that exists. |
|
|
35
35
|
|
|
36
36
|
Customize per-artifact: a pure-function test corpus with no scenario claims reduces N1–N4 to "verified clean — no scenario-named tests in scope"; a path-classifier PR may need N2 exhausted across 8+ canonical inputs.
|
|
37
37
|
|
|
@@ -37,11 +37,11 @@ Decomposition is by the **kind of docstring claim** that needs to be cross-check
|
|
|
37
37
|
| O2 | Fixture docstring vs sibling-test behavior | An autouse / module-scope fixture docstring asserts an invariant (`readability is disabled`, `network is mocked`, `tmp_path is empty`). No sibling test in the same module explicitly opts out of the invariant. |
|
|
38
38
|
| O3 | Predicate-name and -docstring vs body breadth | A boolean helper's name and docstring promise a narrow predicate. Walk the body's branches: every branch's `return True` path is consistent with the promised name. Bodies that accept inputs broader than the name (`_dir_value_resolves_to_shared_temp` also accepting HOME/TMP env-derived paths) are O3 findings. |
|
|
39
39
|
| O4 | Step-ordering narrative | A docstring describes processing as `A then B then C`. Walk the body and confirm the call order matches. Mismatched order is an O4 finding regardless of whether the final output is the same. A docstring step enumeration that names the body's linear steps but omits a corrective workflow step the body guards inside an `if`/`elif` branch (`if not await cancel_and_reinitiate_update(...): return`) is also an O4 finding: the reader trusts the step list to be complete and misses the conditional path. The branch-guarded-dispatch shape of this drift — a docstring that names two or more linear-step callees while the body guards a two-or-more-token dispatch callee inside a branch whose name the prose never spells out — stays judgment for this lane, alongside re-ordered steps and plain unguarded steps the prose omits. |
|
|
40
|
-
| O5 | Named-sentinel / filename references | A docstring names a sentinel marker, environment variable, filename, or magic string. Confirm the named token
|
|
41
|
-
| O6 | Free-form `Args:`-adjacent claims | A docstring's `Returns:` / `Raises:` / `Note:` / `Example:` sections make claims (`returns shared-temp only`, `raises ValueError on missing key`). Verify each claim against the body. When a docstring enumerates the inputs a body counts (a "field counts as read when ..." list, a list of conditions treated as a match, a list of cases the body skips), list every union member and every suppressor the body applies (`read_names = a \| b \| c`, each early-return guard) and confirm each appears in the prose enumeration. A union member or suppressor the body applies but the prose omits is an O6 finding. When a docstring sentence excludes a named category of input from what the function flags (`X are not dispatch steps`, `Y is not a match`), confirm the axis the prose excludes on is the axis the body's branch condition
|
|
40
|
+
| O5 | Named-sentinel / filename references | A docstring names a sentinel marker, environment variable, filename, or magic string. Confirm the named token exists in the module body or in the repo's naming convention. |
|
|
41
|
+
| O6 | Free-form `Args:`-adjacent claims | A docstring's `Returns:` / `Raises:` / `Note:` / `Example:` sections make claims (`returns shared-temp only`, `raises ValueError on missing key`). Verify each claim against the body. When a docstring enumerates the inputs a body counts (a "field counts as read when ..." list, a list of conditions treated as a match, a list of cases the body skips), list every union member and every suppressor the body applies (`read_names = a \| b \| c`, each early-return guard) and confirm each appears in the prose enumeration. A union member or suppressor the body applies but the prose omits is an O6 finding. When a docstring sentence excludes a named category of input from what the function flags (`X are not dispatch steps`, `Y is not a match`), confirm the axis the prose excludes on is the axis the body's branch condition keys on. A body that flags a call when it sits inside an `If.test` guard, paired with prose that excludes by the call's receiver shape (`method-on-local calls inside a branch are not dispatch steps`), is an O6 finding: a guarded method-on-local call is flagged even though the prose lists it as excluded — the exclusion is keyed to the wrong axis. A thin delegating method whose docstring names its actions and points at the home of the body lists the same actions the delegated function's own summary lists; when an edit moves one action out of the delegated body, the same edit rewords both summaries — judgment for this lane. A conditional bullet in the delegated prose also names every exception the body honors — that conditional-completeness slice stays a judgment call for this lane. A `Returns:` that names the mechanism, tool, or output format the function produces (`instructing a StructuredOutput summary`, `returns a YAML document`, `emits a JSON object`) matches the artifact the body builds. A dataclass or `TypedDict` field documented in the class `Attributes:` block states what the field means for one record; when the code sets that field the same way for every record (a run-mode flag such as `is_dry_run = not is_execute`), the description states the run-mode meaning, not a per-record outcome — judgment for this lane. A workflow gate-outcome status flag whose in-code prose (a schema property `description`, an architecture `detail`/overview string) describes the outcome as bypassing or skipping matches the branch that handles it: when the code routes that outcome to a blocker (`blocker = ...; break`) that holds the PR in draft, prose that reads "skips without blocking" or "the gate is bypassed" for that outcome is an O6 finding. The remaining deterministic O6 shape is gated at Write/Edit time — see **Write-time gate inventory** below — so the audit lane focuses on the free-form shapes the gate cannot match. |
|
|
42
42
|
| O7 | Module-doc-vs-split-module after refactor | When a refactor moves a responsibility to a sibling module, the originating module's docstring and the receiving module's docstring both describe the home of that responsibility. A module docstring should describe only the responsibilities it owns. |
|
|
43
43
|
| O8 | Companion-doc ordering/content vs producer | When a PR changes a producer function's ordering or union, read that skill's companion `SKILL.md` and sibling `.md` docs for any sentence naming the same produced artifact (a file path, a JSON key, a named list). A doc sentence that claims the artifact is `sorted` / `alphabetical` / `in sorted order`, or holds `just the at-risk names` / `only the current set`, while the producer merges stored names with new names and appends — preserving file order, not re-sorting the union — is an O8 finding on both counts (wrong order claim, hidden merged-in entries). The finding stands even when the PR diff never touched the `.md` file, because the behavior change orphaned the doc claim. A producer docstring asserting that no consumer reads its output yet (`producer-only artifact`, `no submission-run consumer reads it yet`) is the same companion-doc producer/consumer drift — judgment for this lane. The O6 pattern recurs at the companion-doc layer: when a PR routes a gate outcome to a blocker, read every skill's `SKILL.md` and reference `.md` docs that name that gate for a line still calling the outcome a bypass — an O8 finding. |
|
|
44
|
-
| O9 | Python docstring plainness for a general developer | A changed module / class / public-function docstring's narrative prose — the summary and description before the first `Args:` / `Returns:` / `Raises:` / `Yields:` section — reads plainly and paints a concrete scene a general developer follows on first read. Flag a narrative that stacks abstract machinery nouns into a wall (`the SIGINT install/restore/installability check, the atexit terminal-record registration, and the interrupted-run finalizer`), that defines a thing by what it is not (`the non-promoter-specific machinery`), or that runs one sentence long while joining clauses with an em-dash or a semicolon. The diagram-first shape carries this best: a summary line, then a `::` example block or a doctest that shows a concrete input and its marked outcome, then a couple of short prose lines. The deterministic run-on mark is gated at Write/Edit time by `check_docstring_runon_sentence` in `code_rules_docstrings.py`, and a narrative that runs more than six prose lines with no such block is gated by `check_docstring_prose_wall_without_illustration` in the same module, so this lane carries the judgment the gates cannot: whether a stranger to the code pictures the moment, the input, and the outcome after one read, and whether the diagram
|
|
44
|
+
| O9 | Python docstring plainness for a general developer | A changed module / class / public-function docstring's narrative prose — the summary and description before the first `Args:` / `Returns:` / `Raises:` / `Yields:` section — reads plainly and paints a concrete scene a general developer follows on first read. Flag a narrative that stacks abstract machinery nouns into a wall (`the SIGINT install/restore/installability check, the atexit terminal-record registration, and the interrupted-run finalizer`), that defines a thing by what it is not (`the non-promoter-specific machinery`), or that runs one sentence long while joining clauses with an em-dash or a semicolon. The diagram-first shape carries this best: a summary line, then a `::` example block or a doctest that shows a concrete input and its marked outcome, then a couple of short prose lines. The deterministic run-on mark is gated at Write/Edit time by `check_docstring_runon_sentence` in `code_rules_docstrings.py`, and a narrative that runs more than six prose lines with no such block is gated by `check_docstring_prose_wall_without_illustration` in the same module, so this lane carries the judgment the gates cannot: whether a stranger to the code pictures the moment, the input, and the outcome after one read, and whether the diagram illustrates — an input, a marked outcome, an `ok:` / `flag:` contrast a reader learns from. See `../../rules/plain-illustrative-docstrings.md`. |
|
|
45
45
|
|
|
46
46
|
---
|
|
47
47
|
|
|
@@ -85,7 +85,7 @@ Read the body and the docstring side by side. Apply each check that matches the
|
|
|
85
85
|
- **Suppressor / skip lists.** A body with several early returns that suppress the check names each suppressor in the prose.
|
|
86
86
|
- **Shared fallback routes.** A summary that scopes a fallback call to one condition names every condition that reaches that call — judgment for this lane.
|
|
87
87
|
- **Step order.** A docstring that says `A then B then C` matches the call order in the body. A step enumeration that names the body's linear steps also names every corrective step the body guards inside an `if`/`elif` branch — judgment for this lane.
|
|
88
|
-
- **Delegation pointer summaries.** A thin delegating method whose docstring names its actions and points at the home of the
|
|
88
|
+
- **Delegation pointer summaries.** A thin delegating method whose docstring names its actions and points at the home of the body lists the same actions the delegated function's own summary lists — judgment for this lane, including a conditional bullet in the delegated prose that names every exception the body honors.
|
|
89
89
|
- **JS/`.mjs` resume-task, `@returns` object, sibling return keys, bare-flag directives.** See the JavaScript gate inventory above.
|
|
90
90
|
- **Returns-clause cardinality.** A `Returns:` clause that names a dict-key prefix family with a plural noun matches the count of keys in that family in the returned dict literal — judgment for this lane.
|
|
91
91
|
- **Length-constant superlative vs exact gate.** A module docstring that describes an integer `*_LENGTH` constant with a superlative or range word matches how the code consumes the constant — judgment for this lane.
|
|
@@ -95,7 +95,7 @@ Read the body and the docstring side by side. Apply each check that matches the
|
|
|
95
95
|
- **Module summary scope versus data-schema constants.** A module whose one-line docstring scopes its contents to user-facing text names every category of constant the body holds. Gated form: `check_module_docstring_scope_omits_data_schema_constants`.
|
|
96
96
|
- **Field meaning: run mode versus per record.** A dataclass or `TypedDict` field documented in the class `Attributes:` block states what the field means for one record. When the code sets that field the same way for every record, the description states the run-mode meaning — judgment for this lane.
|
|
97
97
|
- **Predicate breadth.** A boolean helper whose prose promises a narrow check accepts only the inputs the prose names — no broader input class the name and prose do not mention.
|
|
98
|
-
- **Exclusion-clause distinguisher.** A docstring sentence that says a named category of input "are not" / "is not" the thing the function flags keys the exclusion to the same axis the body's classification keys on. Read the body's
|
|
98
|
+
- **Exclusion-clause distinguisher.** A docstring sentence that says a named category of input "are not" / "is not" the thing the function flags keys the exclusion to the same axis the body's classification keys on. Read the body's branch condition, then state the exclusion on that same axis.
|
|
99
99
|
- **Companion-doc ordering and content claims.** A `SKILL.md` (or sibling `.md`) sentence that names a produced artifact and claims its order or its content matches the producer function's docstring and body for that same artifact. The two move together in one commit, even when the producer edit does not touch the `.md` file.
|
|
100
100
|
- **Gate-outcome status flags.** A workflow gate outcome the body routes to a blocker (`blocker = ...; break`) reads as blocked in every in-code prose string (a schema `description`, an architecture `detail`/overview string) and companion doc, never as a bypass — judgment for this lane.
|
|
101
101
|
- **TYPE_CHECKING gate claim vs code.** A docstring that names a `TYPE_CHECKING` gate-detection step matches a module whose code handles TYPE_CHECKING — judgment for this lane.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
**What this category audits:** identifiers and reference data whose label asserts a contract the body does not deliver. The label may be too broad (`is_inside_function` flag set on def but never reset on scope exit), too narrow (`_is_docstring_section_header` matching only terminating headers), or shaped as one thing while behaving as another (`FILE_PATH_PATTERN` regex matching `client/server` because it lacks path-shape anchors; a hard-deny replacement word list containing ordinary technical English).
|
|
4
4
|
|
|
5
|
-
The label-vs-body gap is its own failure mode independent of behavior-equivalence (L) because nothing was rewritten — the contract is broken at the moment the name is first chosen. The hook-enforced naming rules (J5 abbreviations, J6 vague nouns) ban specific identifiers but say nothing about precision-of-fit between what a name promises and what its body
|
|
5
|
+
The label-vs-body gap is its own failure mode independent of behavior-equivalence (L) because nothing was rewritten — the contract is broken at the moment the name is first chosen. The hook-enforced naming rules (J5 abbreviations, J6 vague nouns) ban specific identifiers but say nothing about precision-of-fit between what a name promises and what its body does.
|
|
6
6
|
|
|
7
7
|
**Examples of Category P findings:**
|
|
8
8
|
- A flag `is_inside_function` set on `def` and never reset on scope exit — name asserts state the body fails to keep.
|
|
@@ -25,7 +25,7 @@ For every regex, brace or token scanner, or word-list the diff adds or edits, bu
|
|
|
25
25
|
|
|
26
26
|
Walk each input character by character (or token by token) and write down what the pattern returns. A pattern that returns the wrong answer on any of the three inputs is a Category P finding; cite the input that breaks it. The pattern is clean only after all three inputs return the answer the contract promises.
|
|
27
27
|
|
|
28
|
-
`hooks/blocking/` is the priority surface. Its patterns gate every write, so a pattern that matches too much or too little there fires on
|
|
28
|
+
`hooks/blocking/` is the priority surface. Its patterns gate every write, so a pattern that matches too much or too little there fires on edits across the whole codebase.
|
|
29
29
|
|
|
30
30
|
Two canonical failures show why the hand trace is binding:
|
|
31
31
|
|
|
@@ -56,4 +56,4 @@ The reusable Variant C template for Category P is in [`../prompts/category-p-nam
|
|
|
56
56
|
|
|
57
57
|
## Why Category P matters as its own bucket
|
|
58
58
|
|
|
59
|
-
Category L (behavior-equivalence) audits a rewrite against a prior implementation — it only fires when there is a `before` state to compare. Category P audits a fresh identifier whose label asserts a contract; the bug is that the body never delivered the named contract, even on the first commit. The hook-enforced J5 / J6 naming rules ban specific identifiers (`ctx`, `cfg`, `data`, `result`, `handle_*`) but say nothing about whether the identifier the author chose
|
|
59
|
+
Category L (behavior-equivalence) audits a rewrite against a prior implementation — it only fires when there is a `before` state to compare. Category P audits a fresh identifier whose label asserts a contract; the bug is that the body never delivered the named contract, even on the first commit. The hook-enforced J5 / J6 naming rules ban specific identifiers (`ctx`, `cfg`, `data`, `result`, `handle_*`) but say nothing about whether the identifier the author chose matches the body's reach. P is the bucket that catches a regex named `FILE_PATH_PATTERN` that accepts `TCP/IP`, a hard-deny word list that bans `function` and `address`, and a predicate named for the general case that only handles a subset — at audit time, before the gate ships and starts producing false positives in production.
|
|
@@ -30,7 +30,7 @@ Decomposition is by the **pair of surfaces** that must agree.
|
|
|
30
30
|
| ID | Axis name | Concrete checks |
|
|
31
31
|
|---|---|---|
|
|
32
32
|
| Q1 | Term sweep | Pull out every identifier, API field, and config key the diff introduces or reads. Sweep all changed prose surfaces for near-miss variants — hyphen or space in place of an underscore, singular where the code is plural (or the reverse), a shared prefix with a divergent tail. Run `packages/claude-dev-env/_shared/pr-loop/scripts/terminology_sweep.py` for mechanical recall — it reads a unified diff on stdin or via `--diff-file`, prints one `file:line` finding per near-miss, exits 1 when findings exist and 0 when clean — then adjudicate each finding, since a near-miss can be an intentional distinct term. |
|
|
33
|
-
| Q2 | PR-description claim verification | Fetch the PR body. For each falsifiable claim, verify it two ways against the diff and the repo. A dead-code claim ("removes the last caller of X") gets a reference search across the repo. A performance or ceiling claim ("caps at 30 seconds", "cuts calls in half") gets checked against the
|
|
33
|
+
| Q2 | PR-description claim verification | Fetch the PR body. For each falsifiable claim, verify it two ways against the diff and the repo. A dead-code claim ("removes the last caller of X") gets a reference search across the repo. A performance or ceiling claim ("caps at 30 seconds", "cuts calls in half") gets checked against the config values and call sites the diff sets. |
|
|
34
34
|
| Q3 | Message-vs-guard consistency | For every log, error, and evidence string the diff adds or edits, read the condition that gates it. The message text matches what the guard guarantees — a string that asserts an outcome fires only on a branch that reaches that outcome. Temporal wording matches the lifetime of the state variable it names: "this run" versus "this round", "is open" versus "was opened", "reached" versus "attempted". |
|
|
35
35
|
|
|
36
36
|
Customize per-artifact: a change that adds no new identifier and no PR-body claim reduces Q to "verify every message the diff touches still matches its guard." A rename that spreads a term across many docs may need Q1 alone to list every prose surface.
|
|
@@ -20,7 +20,7 @@ ID prefix: `find`.
|
|
|
20
20
|
|
|
21
21
|
**A2. Return-type annotation vs every code path**
|
|
22
22
|
- For each annotated function, walk every code path: explicit `return X`, fall-through to implicit `None`, exception-handler exit, generator `yield` paths, async coroutine return value.
|
|
23
|
-
- Verify each path's
|
|
23
|
+
- Verify each path's return value is assignable to the declared annotation; flag `-> bool` functions that can return `None`, `-> list[T]` functions that can return `None` on an early exit, etc.
|
|
24
24
|
- For functions that raise instead of returning on some path, confirm the annotation does not promise a value the caller will dereference.
|
|
25
25
|
- Inspect `try/except/finally` chains for paths that return from `finally` and override `try`/`except` returns.
|
|
26
26
|
- For async functions, confirm the annotation refers to the awaited type, not the coroutine wrapper.
|
|
@@ -46,7 +46,7 @@ ID prefix: `find`.
|
|
|
46
46
|
- Verify kwargs are valid for the targeted runtime version (`capture_output`, `text`, `encoding`, `check`, `timeout`); flag combinations that conflict (`stdout=PIPE` + `capture_output=True`).
|
|
47
47
|
- Exception contract under `check=True` (raises `CalledProcessError` on non-zero exit) — verify callers either propagate or handle, with no silent swallow that masks failure.
|
|
48
48
|
- Verify quoting/escaping of arguments crossing the subprocess boundary, especially when interpolating untrusted strings.
|
|
49
|
-
- Verify the resolved executable path
|
|
49
|
+
- Verify the resolved executable path exists on the target platform, not assumed (`which` / `Get-Command` failure paths).
|
|
50
50
|
|
|
51
51
|
**A6. Shell/host-language cmdlet or function parameter sets and binding**
|
|
52
52
|
- For every shell or host-language function/cmdlet declaration with parameter sets (PowerShell `param(...)` with `ParameterSetName=`, Bash `getopts`, etc.), verify the declaration matches every invocation pattern. Confirm a default parameter set is declared if no-arg invocation is reachable.
|
|
@@ -11,8 +11,8 @@ ID prefix: `find`.
|
|
|
11
11
|
|
|
12
12
|
**B1. CSS / DOM selector vs target browser engine**
|
|
13
13
|
- Every CSS selector in the diff — verify pseudo-class support (`:has()`, `:is()`, `:where()`, `:focus-visible`, `:focus-within`) against every browser engine in the declared support matrix; flag any selector that requires an engine version newer than the declared minimum.
|
|
14
|
-
- Every attribute selector, `::part()`, `::slotted()`, and shadow-DOM piercing pattern — verify the target engine
|
|
15
|
-
- Every selector fed to `document.querySelector` / `querySelectorAll` / jQuery `$()` / Selenium `By.css_selector` / Playwright `page.locator` — verify the **runtime** selector engine matches the **target browser** engine; a Node-side jsdom selector parse can succeed where the
|
|
14
|
+
- Every attribute selector, `::part()`, `::slotted()`, and shadow-DOM piercing pattern — verify the target engine exposes the matching DOM (e.g. shadow boundaries, scoped styles).
|
|
15
|
+
- Every selector fed to `document.querySelector` / `querySelectorAll` / jQuery `$()` / Selenium `By.css_selector` / Playwright `page.locator` — verify the **runtime** selector engine matches the **target browser** engine; a Node-side jsdom selector parse can succeed where the browser fails (or vice versa).
|
|
16
16
|
- Every CSS feature query (`@supports`), media query level, and container query — verify availability across the declared engine matrix and that fallback paths exist for engines that do not parse the syntax.
|
|
17
17
|
- Every test-runner DOM assertion (snapshot, `outerHTML`/`innerHTML` equality, computed-style read) — verify the runner uses the same engine as the production target, or document the divergence.
|
|
18
18
|
- Cross-engine quirks: WebKit-only `-webkit-` prefixes, Gecko-only `-moz-`, IE/Edge legacy filters; flag any production reliance on a single-engine prefix without a standard fallback.
|
|
@@ -35,7 +35,7 @@ ID prefix: `find`.
|
|
|
35
35
|
|
|
36
36
|
**B4. Shell / CLI / cmdlet syntax vs runtime version**
|
|
37
37
|
- Every PowerShell cmdlet — verify availability across the declared edition matrix (Windows PowerShell 5.1 ↔ PowerShell 7+ ↔ PowerShell on Linux/macOS) and version-specific parameter sets.
|
|
38
|
-
- Every shebang (`#!/usr/bin/env pwsh`, `#!/bin/bash`, `#!/usr/bin/env python3`) vs the
|
|
38
|
+
- Every shebang (`#!/usr/bin/env pwsh`, `#!/bin/bash`, `#!/usr/bin/env python3`) vs the interpreter resolved at runtime — flag mismatches between declared and invoked interpreter (e.g. shebang says `pwsh` but Python `subprocess.run(["powershell", …])` resolves to PS 5.1 on Windows).
|
|
39
39
|
- Every parameter set in a `param(...)` block — verify `[CmdletBinding(DefaultParameterSetName=…)]` is set when ambiguity is possible; missing default = `Parameter set cannot be resolved` at runtime.
|
|
40
40
|
- Every cmdlet flag combination — verify both flags belong to the same parameter set per Microsoft docs (e.g. `-RepetitionInterval` is an `-Once` parameter, NOT `-Daily`).
|
|
41
41
|
- Every bash-ism — `[[ ... ]]` (bash 3+ only, not POSIX `sh`), arrays, process substitution `<(…)`, `${var,,}` lower-case expansion (bash 4+), associative arrays — verify against the declared minimum shell.
|
|
@@ -21,7 +21,7 @@ Inline the artifact under review here (full diff, full file bodies, or a faithfu
|
|
|
21
21
|
- For each site, identify the corresponding release: `with`/context manager, `try`/`finally` `close()`, RAII destructor, `using` block, deferred close, explicit `.close()` reachable on every code path including early `return`, raised exception, and `break`/`continue`.
|
|
22
22
|
- Single-call helpers that open-write-close internally (e.g. `pathlib.Path.write_text`, `pathlib.Path.read_bytes`, `fs.writeFileSync`) do not return a handle to user code and are not a leak site.
|
|
23
23
|
- Distinguish file objects from file descriptors: a wrapped FD (`os.fdopen`) is owned by the file object and closed when the object closes; a raw FD from `os.open` belongs to C8.
|
|
24
|
-
- Shape B proof-of-absence requires ≥3 adversarial probes specific to this sub-bucket. Examples: (a) does any code path open a file via stdlib (`open`, `io.open`, `os.fdopen`, `pathlib.Path.open`, language-equivalent)? (b) do any error/exception callbacks (e.g. an `onerror=` handler) ever receive an object carrying an open FD that the callback must close? (c) do any helper APIs that look like single-call helpers
|
|
24
|
+
- Shape B proof-of-absence requires ≥3 adversarial probes specific to this sub-bucket. Examples: (a) does any code path open a file via stdlib (`open`, `io.open`, `os.fdopen`, `pathlib.Path.open`, language-equivalent)? (b) do any error/exception callbacks (e.g. an `onerror=` handler) ever receive an object carrying an open FD that the callback must close? (c) do any helper APIs that look like single-call helpers return a handle the caller is expected to close?
|
|
25
25
|
|
|
26
26
|
**C2. Subprocess / child processes**
|
|
27
27
|
- Enumerate every site that spawns a child process: `subprocess.run`, `subprocess.Popen`, `os.system`, `os.spawn*`, `multiprocessing.Process`, `child_process.spawn`, `child_process.exec`, `Process.Start`, `os/exec.Command`, shell-out wrappers.
|
|
@@ -44,8 +44,8 @@ Inline the artifact under review here (full diff, full file bodies, or a faithfu
|
|
|
44
44
|
- For each site, verify a release path on every exit branch: explicit `close()`, `with` block, connection-pool return, `dispose`/`destroy`, `defer client.Close()`.
|
|
45
45
|
- Pooled clients have their own lifecycle: the pool itself must be closed on application shutdown, and individual borrows must be returned even on exception.
|
|
46
46
|
- Side-channel connections introduced by libraries (telemetry, metrics, tracing) count too if they are configured by code in the diff.
|
|
47
|
-
- Local-only invocations of cmdlets, CLIs, or RPC endpoints should be distinguished from
|
|
48
|
-
- Shape B proof-of-absence requires ≥3 adversarial probes. Examples: (a) does any imported stdlib or third-party module implicitly open a network socket on import, on first use, or at module construction time? (b) do any local cmdlet / CLI invocations
|
|
47
|
+
- Local-only invocations of cmdlets, CLIs, or RPC endpoints should be distinguished from network calls.
|
|
48
|
+
- Shape B proof-of-absence requires ≥3 adversarial probes. Examples: (a) does any imported stdlib or third-party module implicitly open a network socket on import, on first use, or at module construction time? (b) do any local cmdlet / CLI invocations reach a remote service (e.g. cloud-provider auth lookup, package-registry resolution, AppX execution-alias) under common configurations? (c) do any name-resolution or service-discovery calls (DNS, mDNS, AppX alias, `Get-Command`) hit network paths under common Windows / macOS / Linux configurations?
|
|
49
49
|
|
|
50
50
|
**C5. Locks, semaphores, mutexes**
|
|
51
51
|
- Enumerate every synchronization primitive constructed: `threading.Lock`, `threading.RLock`, `threading.Semaphore`, `threading.BoundedSemaphore`, `threading.Event`, `threading.Condition`, `multiprocessing.Lock`/`Semaphore`/`Manager` locks, `asyncio.Lock`/`Semaphore`/`Event`/`Condition`, advisory file locks (`fcntl.flock`, `msvcrt.locking`), DB advisory locks, distributed locks.
|
|
@@ -65,7 +65,7 @@ Inline the artifact under audit using one `###` header per section. Pick the chu
|
|
|
65
65
|
|
|
66
66
|
**D7. Async/sync ordering of side effects**
|
|
67
67
|
- Scan the entire artifact for `async def`, `await`, `asyncio.gather` / `asyncio.create_task` / `asyncio.run`, JavaScript `async` / `await` / `Promise.all`, or any other deferred-execution primitive. If any are present, walk the ordering of side effects.
|
|
68
|
-
- For each `await` site, identify whether a side effect that should have happened *before* the suspension point is
|
|
68
|
+
- For each `await` site, identify whether a side effect that should have happened *before* the suspension point is flushed before yielding control. Probe what an interleaved coroutine could observe.
|
|
69
69
|
- For purely synchronous artifacts, cite proof-of-absence with explicit keyword counts ("0 occurrences of `async`, `await`, `asyncio` across all [N] sections").
|
|
70
70
|
|
|
71
71
|
**D8. Class-attribute vs instance-attribute confusion**
|
|
@@ -26,7 +26,7 @@ Inline the artifact under this section using the section types defined in the ch
|
|
|
26
26
|
|
|
27
27
|
**E2. Functions / methods defined but never called**
|
|
28
28
|
- For every function, method, or callable defined or modified by the artifact, enumerate call sites: direct calls, callback registrations (`onerror=`, `key=`, signal handlers), decorator applications, `__main__` guards, framework-driven discovery (pytest collection, Django URL resolution, FastAPI route decorators, Click groups), and string-form references resolved at runtime (`getattr`, dispatch tables).
|
|
29
|
-
- Leading-underscore names: a single internal call site is
|
|
29
|
+
- Leading-underscore names: a single internal call site is enough; explicitly verify nothing outside the file imports the underscore-prefixed name.
|
|
30
30
|
- Public names: confirm at least one call site inside or outside the file, or confirm the function is part of a documented public API.
|
|
31
31
|
- Framework-discovered callables (test functions, route handlers, CLI commands): state which collector picks them up and confirm the artifact's path matches that collector's pattern.
|
|
32
32
|
|
|
@@ -49,7 +49,7 @@ Inline the artifact under this section using the section types defined in the ch
|
|
|
49
49
|
- Tuple-unpack discards (`for path, _, _ in os.walk(...)`) are out of scope — E5 specifically scopes "function parameters never read"; state this exclusion explicitly.
|
|
50
50
|
- `*args` / `**kwargs` / TypeScript rest spreads: confirm at least one consumer (forwarded to another call, iterated, indexed) or mark the parameter unused.
|
|
51
51
|
- Cross-language parameter declarations (PowerShell `param(...)`, shell positional `$1..$N`, Bash `getopts`): confirm each named parameter has at least one body reference.
|
|
52
|
-
- Adversarial probes for proof-of-absence: (a) any test fixture parameter (e.g., `def test_x(tmp_path):`) declared but never used? (b) any non-Python script parameter declared but never referenced? (c) any CLI flag parsed by argparse / Click / Cobra but unreachable on every invocation path the artifact
|
|
52
|
+
- Adversarial probes for proof-of-absence: (a) any test fixture parameter (e.g., `def test_x(tmp_path):`) declared but never used? (b) any non-Python script parameter declared but never referenced? (c) any CLI flag parsed by argparse / Click / Cobra but unreachable on every invocation path the artifact uses?
|
|
53
53
|
|
|
54
54
|
**E6. Removed-but-not-deleted symbol references**
|
|
55
55
|
- If the artifact deletes or renames a symbol, confirm every import, call site, string reference, and docstring/comment mentioning the old name has been updated or removed.
|
|
@@ -71,7 +71,7 @@ Inline the artifact under this section using the section types defined in the ch
|
|
|
71
71
|
|
|
72
72
|
**E9. Constants-module exports with no importer**
|
|
73
73
|
- For every module-level `UPPER_SNAKE` constant the artifact adds to a `*_constants.py` or `config/` module, grep the whole repo for the constant name and locate at least one importer (`from <module> import <NAME>`) or in-file reference.
|
|
74
|
-
-
|
|
74
|
+
- Every name a constants module exports carries zero in-file references by design and no write-time check reads such a module, so a dead export reaches the audit untouched; this sub-bucket is the audit-time backstop.
|
|
75
75
|
- A sibling that a consumer module imports is live; a constant that no `from ... import` line and no in-file reference names anywhere in the repo is dead and must be removed (CODE_RULES 9.8).
|
|
76
76
|
- Constants reached only by string-form lookup (`getattr(config, name)`, settings registries) are live; name the dynamic consumer when you mark such a constant referenced.
|
|
77
77
|
- Adversarial probes for proof-of-absence: (a) does the artifact add any constant to a `*_constants.py` / `config/` module whose name returns zero hits outside its own definition line? (b) is any newly added constant shadowed by a same-named constant in a sibling module so the importer resolves the other one? (c) does any constant exist only as an `__all__` re-export with no downstream importer of that re-export?
|
|
@@ -88,7 +88,7 @@ Q3: Which symbol most likely will *become* dead code after a near-future refacto
|
|
|
88
88
|
|
|
89
89
|
Lead: `Total: N (P0=N, P1=N, P2=N)`. For each sub-bucket E1-E9, produce Shape A or Shape B (with ≥3 probes). Cross-bucket Q1-Q3 answers after the per-sub-bucket walk. Adversarial second pass: "assume your first pass missed at least 3 P2 dead-code instances across these 9 sub-buckets — find them." Open Questions section for ambiguities. Read-only. No edits, no commits.
|
|
90
90
|
|
|
91
|
-
Note: most Category E findings are P2 (style / cleanup) unless the dead code masks
|
|
91
|
+
Note: most Category E findings are P2 (style / cleanup) unless the dead code masks a bug; the adversarial-pass quota uses P2 here.
|
|
92
92
|
|
|
93
93
|
---
|
|
94
94
|
|
|
@@ -184,7 +184,7 @@ Q3: Which symbol most likely will *become* dead code after a near-future refacto
|
|
|
184
184
|
|
|
185
185
|
Lead: `Total: N (P0=N, P1=N, P2=N)`. For each sub-bucket E1-E9, produce Shape A or Shape B (with ≥3 probes). Cross-bucket Q1-Q3 answers after the per-sub-bucket walk. Adversarial second pass: "assume your first pass missed at least 3 P2 dead-code instances across these 9 sub-buckets — find them." Open Questions section for ambiguities. Read-only. No edits, no commits.
|
|
186
186
|
|
|
187
|
-
Note: most Category E findings are P2 (style / cleanup) unless the dead code masks
|
|
187
|
+
Note: most Category E findings are P2 (style / cleanup) unless the dead code masks a bug; the adversarial-pass quota uses P2 here.
|
|
188
188
|
|
|
189
189
|
## Diff (4 new files, all lines in scope)
|
|
190
190
|
|