claude-dev-env 8.60.0 → 8.61.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/pr-lifecycle/SKILL.md +5 -0
- package/bin/merge_profile_settings.mjs +6 -0
- package/docs/rule-guides/proof-before-pull-request.md +37 -0
- package/hooks/blocking/followup_pr_dedupe.py +7 -2
- package/hooks/blocking/pr_lifecycle_skill_gate.py +5 -1
- package/hooks/blocking/pull_request_proof.py +141 -0
- package/hooks/blocking/test_followup_pr_dedupe.py +8 -0
- package/hooks/blocking/test_pr_lifecycle_skill_gate.py +13 -4
- package/hooks/blocking/test_pull_request_proof.py +92 -0
- package/hooks/hooks_constants/pull_request_proof_constants.py +27 -0
- package/package.json +1 -1
- package/rules/proof-before-pull-request.md +7 -0
- package/rules/question-presentation.md +4 -7
- package/scripts/tests/test_rule_load_scopes.py +1 -0
|
@@ -22,6 +22,11 @@ User-level rule: applies to **every** git repo that uses GitHub with `gh`. Small
|
|
|
22
22
|
|
|
23
23
|
### Pull request submission rules
|
|
24
24
|
|
|
25
|
+
**Prove the change where it runs before you open the pull request.** The body carries a
|
|
26
|
+
"Proof in practice" section with each command you ran and its quoted output. A hook denies
|
|
27
|
+
a new pull request without one. Read `~/.claude/docs/rule-guides/proof-before-pull-request.md`
|
|
28
|
+
before you plan the proof run.
|
|
29
|
+
|
|
25
30
|
**Open every pull request ready for review.** Pass `--draft` only when the owner asks
|
|
26
31
|
for a draft.
|
|
27
32
|
|
|
@@ -65,6 +65,12 @@ export const DECLARED_PROFILE_SETTINGS = Object.freeze([
|
|
|
65
65
|
+ '(mcp__claude-code-remote__set_session_title or mcp__ccd_session_mgmt__set_session_title) is allowed, '
|
|
66
66
|
+ 'and so is a live test run of the account broker with '
|
|
67
67
|
+ '`python ~/.claude/scripts/account_broker.py run`: the user installed both tools for these sessions',
|
|
68
|
+
'Moving instruction files out of the always-loaded rules folder of a Claude config directory '
|
|
69
|
+
+ '(`~/.claude/rules` or a profile\'s `rules` folder) into a sibling on-demand folder in the same '
|
|
70
|
+
+ 'config directory, and moving them back, is allowed when the task is to trim session-start context: '
|
|
71
|
+
+ 'each file keeps its contents and stays on disk, and the move reverses with one more move. '
|
|
72
|
+
+ 'This covers a plain move or rename with an index file that says when to open each moved file. '
|
|
73
|
+
+ 'It does not cover deleting or editing a rule, or any change to settings, hooks or permissions.',
|
|
68
74
|
],
|
|
69
75
|
},
|
|
70
76
|
]);
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
Back to the [rule entry](../../rules/proof-before-pull-request.md).
|
|
2
|
+
|
|
3
|
+
# Proof before a pull request
|
|
4
|
+
|
|
5
|
+
**When this applies:** Every session that builds a change and opens a pull request, in every project.
|
|
6
|
+
|
|
7
|
+
## Rule
|
|
8
|
+
|
|
9
|
+
Before you open a pull request, prove the change does its job where it will run.
|
|
10
|
+
|
|
11
|
+
What counts as proof:
|
|
12
|
+
- You ran the changed thing the way its user runs it, and you saw the effect it exists to produce. A hook fires in a live Claude Code session and changes what that session does. A script runs from its launcher on inputs like the ones it will meet. A skill or rule changes what a fresh agent does with an ordinary request.
|
|
13
|
+
- You saw the failure first. Run the same scenario without your change and record what goes wrong. The proof is the difference between the two runs.
|
|
14
|
+
- You covered the conditions the change will meet. That includes the case it should leave alone, and for session behavior a fresh session, a resumed session, a later turn, and a session after compaction.
|
|
15
|
+
- The session under test got an ordinary request, worded as a task the way Jon would ask for it.
|
|
16
|
+
|
|
17
|
+
Unit tests are welcome as extra evidence. The live run is the proof. A test you edited until it passed proves nothing.
|
|
18
|
+
|
|
19
|
+
For a change to Claude Code behavior, such as a hook, skill, rule, setting, prompt or output style:
|
|
20
|
+
- Spawn every session through the account broker, with the stream saved to a file:
|
|
21
|
+
python ~/.claude/scripts/account_broker.py run --product claude --report <broker-report.json> -- claude -p "<request>" --model claude-opus-5-5 --effort <level> --output-format stream-json --verbose > <stream.jsonl>
|
|
22
|
+
- Run each later turn the same way with `--resume <session id>`. The broker resumes a session on the account that started it when that account has room.
|
|
23
|
+
- Keep `--model` and `--effort` on every call. Without them the headless run uses claude-sonnet-5-5. Confirm the served model in the result's `modelUsage`.
|
|
24
|
+
- Start the broker from your own shell, with your environment as is. The broker reads the account meters with your session's credentials and starts Claude without your session's variables.
|
|
25
|
+
- Leave out `--bare` and `claude plugin eval`. Both drop the hooks and settings the user runs with.
|
|
26
|
+
- Read the broker report for the account it chose. Read the stream for the hook events and their output, `permission_denials`, the final result, and the token usage on the result event.
|
|
27
|
+
- When the broker exits 3 with action "wait", no account has room. Treat that as a blocked proof and report its reason and `resets_at`. Spawn Claude only through the broker.
|
|
28
|
+
|
|
29
|
+
Report it in the pull request body under "Proof in practice". List each command you ran, quote the output lines that show the effect, state the difference between the runs with and without the change, and name anything still unproven.
|
|
30
|
+
|
|
31
|
+
When you cannot prove it, leave the pull request unopened. Tell Jon in three short sentences what you could not run and which tool would make it possible. A blocked proof is an acceptable result, because it shows where tooling is missing.
|
|
32
|
+
|
|
33
|
+
## Enforcement
|
|
34
|
+
|
|
35
|
+
`hooks/blocking/pull_request_proof.py` reads the body of each new pull request before it opens. `pr_lifecycle_skill_gate.py` runs it on `gh pr create`, the `pull_request.py create` script, and each `mcp__*__create_pull_request` tool. The gate denies the call when the body has no "Proof in practice" heading, or when the text under that heading names no command in backticks. A `gh pr create` with no `--body` or `--body-file` is denied with a request for the body. Edits, merges, and pushes pass.
|
|
36
|
+
|
|
37
|
+
The gate checks that the section is there and names a command. Whether the run proves the change stays a judgment for the author and the reviewer.
|
|
@@ -79,7 +79,12 @@ def _option_value(all_arguments: list[str], all_option_names: frozenset[str]) ->
|
|
|
79
79
|
return None
|
|
80
80
|
|
|
81
81
|
|
|
82
|
-
def
|
|
82
|
+
def body_from_arguments(all_arguments: list[str], working_directory: str) -> str | None:
|
|
83
|
+
"""Return the body a gh-style argument list passes, or None when it passes none.
|
|
84
|
+
|
|
85
|
+
``--body`` text comes back as given. A ``--body-file`` path is read from
|
|
86
|
+
the working directory. Standard input and an unreadable file give None.
|
|
87
|
+
"""
|
|
83
88
|
inline_body = _option_value(all_arguments, ALL_BODY_OPTIONS)
|
|
84
89
|
if inline_body is not None:
|
|
85
90
|
return inline_body
|
|
@@ -125,7 +130,7 @@ def _segment_draft(all_segment_tokens: list[str], working_directory: str) -> Pul
|
|
|
125
130
|
program, all_arguments = segment_program_and_arguments(all_segment_tokens)
|
|
126
131
|
if program != "gh" or all_arguments[: len(ALL_GH_CREATE_WORDS)] != ALL_GH_CREATE_WORDS:
|
|
127
132
|
return None
|
|
128
|
-
body =
|
|
133
|
+
body = body_from_arguments(all_arguments, working_directory)
|
|
129
134
|
repository = _repository_from_arguments(all_arguments, working_directory)
|
|
130
135
|
if body is None or repository is None:
|
|
131
136
|
return None
|
|
@@ -53,6 +53,7 @@ from hooks_constants.hook_specific_output_keys import (
|
|
|
53
53
|
from hooks_constants.pre_tool_use_stdin import read_hook_input_dictionary_from_stdin
|
|
54
54
|
from hooks_constants.setup_project_paths_constants import DECODE_ERRORS_POLICY, UTF8_ENCODING
|
|
55
55
|
from blocking.followup_pr_dedupe import duplicate_followup_reason
|
|
56
|
+
from blocking.pull_request_proof import missing_proof_reason
|
|
56
57
|
from transcript_skill_scan import is_skill_loaded_after_last_compaction
|
|
57
58
|
|
|
58
59
|
|
|
@@ -198,7 +199,7 @@ def _subagent_transcript_path(all_payload_fields: dict[str, object]) -> str | No
|
|
|
198
199
|
|
|
199
200
|
|
|
200
201
|
def decision_for(all_payload_fields: dict[str, object]) -> dict[str, object] | None:
|
|
201
|
-
"""Return a deny for an unloaded skill or a second follow-up
|
|
202
|
+
"""Return a deny for an unloaded skill, a new pull request with no proof, or a second follow-up.
|
|
202
203
|
|
|
203
204
|
Args:
|
|
204
205
|
all_payload_fields: The parsed PreToolUse input.
|
|
@@ -207,6 +208,9 @@ def decision_for(all_payload_fields: dict[str, object]) -> dict[str, object] | N
|
|
|
207
208
|
return None
|
|
208
209
|
if _is_skill_unloaded(all_payload_fields):
|
|
209
210
|
return _deny(DENY_REASON)
|
|
211
|
+
proof_reason = missing_proof_reason(all_payload_fields)
|
|
212
|
+
if proof_reason is not None:
|
|
213
|
+
return _deny(proof_reason)
|
|
210
214
|
duplicate_reason = duplicate_followup_reason(all_payload_fields)
|
|
211
215
|
return None if duplicate_reason is None else _deny(duplicate_reason)
|
|
212
216
|
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
"""Deny a new pull request whose body carries no 'Proof in practice' section."""
|
|
2
|
+
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import re
|
|
6
|
+
import sys
|
|
7
|
+
from collections.abc import Mapping
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
|
|
10
|
+
hooks_directory = str(Path(__file__).resolve().parent.parent)
|
|
11
|
+
if hooks_directory not in sys.path:
|
|
12
|
+
sys.path.insert(0, hooks_directory)
|
|
13
|
+
|
|
14
|
+
from blocking.followup_pr_dedupe import body_from_arguments
|
|
15
|
+
from hooks_constants.pr_lifecycle_skill_gate_constants import SHELL_TOOL_NAMES
|
|
16
|
+
from hooks_constants.pull_request_proof_constants import (
|
|
17
|
+
ALL_GH_CREATE_WORDS,
|
|
18
|
+
ALL_PYTHON_PROGRAM_NAMES,
|
|
19
|
+
COMMAND_MARKER,
|
|
20
|
+
CREATE_PULL_REQUEST_TOOL_SUFFIX,
|
|
21
|
+
LINE_SEPARATOR,
|
|
22
|
+
MISSING_BODY_REASON,
|
|
23
|
+
MISSING_PROOF_REASON,
|
|
24
|
+
NEXT_HEADING_PATTERN,
|
|
25
|
+
PROOF_HEADING_PATTERN,
|
|
26
|
+
PULL_REQUEST_SCRIPT_CREATE_WORD,
|
|
27
|
+
PULL_REQUEST_SCRIPT_NAME,
|
|
28
|
+
)
|
|
29
|
+
from hooks_constants.pytest_invocation import unquoted_token
|
|
30
|
+
from hooks_constants.shell_command_pipeline import pipeline_segments_for_command
|
|
31
|
+
from hooks_constants.shell_command_segments import token_basename
|
|
32
|
+
from hooks_constants.shell_command_wrappers import (
|
|
33
|
+
all_wrapped_command_texts,
|
|
34
|
+
segment_program_and_arguments,
|
|
35
|
+
)
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def _heading_level(line: str, heading_pattern: str) -> int | None:
|
|
39
|
+
heading = re.match(heading_pattern, line.strip(), re.IGNORECASE)
|
|
40
|
+
return None if heading is None else len(heading.group(1))
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def _lines_until_heading(all_lines: list[str], heading_level: int) -> list[str]:
|
|
44
|
+
for each_index, each_line in enumerate(all_lines):
|
|
45
|
+
next_level = _heading_level(each_line, NEXT_HEADING_PATTERN)
|
|
46
|
+
if next_level is not None and next_level <= heading_level:
|
|
47
|
+
return all_lines[:each_index]
|
|
48
|
+
return all_lines
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def proof_section(body: str) -> str | None:
|
|
52
|
+
"""Return the text under the body's 'Proof in practice' heading, or None.
|
|
53
|
+
|
|
54
|
+
::
|
|
55
|
+
|
|
56
|
+
"## Proof in practice\\nran `x`\\n## Notes" -> "ran `x`"
|
|
57
|
+
"## Summary\\nno proof heading" -> None
|
|
58
|
+
|
|
59
|
+
The section runs to the next heading of the same or a higher level.
|
|
60
|
+
"""
|
|
61
|
+
all_lines = body.splitlines()
|
|
62
|
+
for each_index, each_line in enumerate(all_lines):
|
|
63
|
+
heading_level = _heading_level(each_line, PROOF_HEADING_PATTERN)
|
|
64
|
+
if heading_level is not None:
|
|
65
|
+
all_section_lines = _lines_until_heading(all_lines[each_index + 1 :], heading_level)
|
|
66
|
+
return LINE_SEPARATOR.join(all_section_lines).strip()
|
|
67
|
+
return None
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def _script_arguments(program: str, all_arguments: list[str]) -> list[str] | None:
|
|
71
|
+
if program == PULL_REQUEST_SCRIPT_NAME:
|
|
72
|
+
return all_arguments
|
|
73
|
+
if program not in ALL_PYTHON_PROGRAM_NAMES or not all_arguments:
|
|
74
|
+
return None
|
|
75
|
+
script_name = token_basename(unquoted_token(all_arguments[0]))
|
|
76
|
+
return all_arguments[1:] if script_name == PULL_REQUEST_SCRIPT_NAME else None
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
def _script_create_arguments(program: str, all_arguments: list[str]) -> list[str] | None:
|
|
80
|
+
all_script_arguments = _script_arguments(program, all_arguments)
|
|
81
|
+
if all_script_arguments is None or all_script_arguments[:1] != [PULL_REQUEST_SCRIPT_CREATE_WORD]:
|
|
82
|
+
return None
|
|
83
|
+
return all_script_arguments[1:]
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
def _create_arguments(all_segment_tokens: list[str]) -> list[str] | None:
|
|
87
|
+
program, all_arguments = segment_program_and_arguments(all_segment_tokens)
|
|
88
|
+
if program == "gh" and all_arguments[: len(ALL_GH_CREATE_WORDS)] == ALL_GH_CREATE_WORDS:
|
|
89
|
+
return all_arguments[len(ALL_GH_CREATE_WORDS) :]
|
|
90
|
+
return _script_create_arguments(program, all_arguments)
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
def _shell_create_bodies(command: str, working_directory: str) -> list[str | None]:
|
|
94
|
+
return [
|
|
95
|
+
body_from_arguments(all_create_arguments, working_directory)
|
|
96
|
+
for each_text in all_wrapped_command_texts(command)
|
|
97
|
+
for each_segment, _each_following_operator in pipeline_segments_for_command(each_text)
|
|
98
|
+
if (all_create_arguments := _create_arguments(each_segment)) is not None
|
|
99
|
+
]
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
def _create_bodies(all_payload_fields: Mapping[str, object]) -> list[str | None]:
|
|
103
|
+
tool_name = all_payload_fields.get("tool_name")
|
|
104
|
+
tool_input = all_payload_fields.get("tool_input")
|
|
105
|
+
if not isinstance(tool_name, str) or not isinstance(tool_input, dict):
|
|
106
|
+
return []
|
|
107
|
+
if tool_name.endswith(CREATE_PULL_REQUEST_TOOL_SUFFIX):
|
|
108
|
+
body = tool_input.get("body")
|
|
109
|
+
return [body if isinstance(body, str) else ""]
|
|
110
|
+
command = tool_input.get("command")
|
|
111
|
+
if tool_name not in SHELL_TOOL_NAMES or not isinstance(command, str):
|
|
112
|
+
return []
|
|
113
|
+
working_directory = all_payload_fields.get("cwd")
|
|
114
|
+
return _shell_create_bodies(
|
|
115
|
+
command, working_directory if isinstance(working_directory, str) else ""
|
|
116
|
+
)
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
120
|
+
"""Return a deny reason when a new pull request's body has no proof, else None.
|
|
121
|
+
|
|
122
|
+
::
|
|
123
|
+
|
|
124
|
+
gh pr create --body-file body.md, body has the section -> None
|
|
125
|
+
mcp__github__create_pull_request, body without it -> MISSING_PROOF_REASON
|
|
126
|
+
gh pr create --fill -> MISSING_BODY_REASON
|
|
127
|
+
gh pr edit, git push, any other tool -> None
|
|
128
|
+
|
|
129
|
+
The proof section is a 'Proof in practice' heading whose text names at
|
|
130
|
+
least one command in backticks.
|
|
131
|
+
|
|
132
|
+
Args:
|
|
133
|
+
all_payload_fields: The parsed PreToolUse input.
|
|
134
|
+
"""
|
|
135
|
+
for each_body in _create_bodies(all_payload_fields):
|
|
136
|
+
if each_body is None:
|
|
137
|
+
return MISSING_BODY_REASON
|
|
138
|
+
section = proof_section(each_body)
|
|
139
|
+
if not section or COMMAND_MARKER not in section:
|
|
140
|
+
return MISSING_PROOF_REASON
|
|
141
|
+
return None
|
|
@@ -84,3 +84,11 @@ def test_powershell_body_file_is_denied(tmp_path: Path) -> None:
|
|
|
84
84
|
}
|
|
85
85
|
reason = dedupe.duplicate_followup_reason(payload, _reader([OPEN_FOLLOWUP]))
|
|
86
86
|
assert reason is not None and "pull request #1769" in reason
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
def test_body_from_arguments_reads_inline_text_and_body_files(tmp_path: Path) -> None:
|
|
90
|
+
(tmp_path / "body.md").write_text("From the file.", encoding="utf-8")
|
|
91
|
+
assert dedupe.body_from_arguments(["--body", "Inline."], "") == "Inline."
|
|
92
|
+
assert dedupe.body_from_arguments(["--body-file", "body.md"], str(tmp_path)) == "From the file."
|
|
93
|
+
assert dedupe.body_from_arguments(["--body-file", "-"], str(tmp_path)) is None
|
|
94
|
+
assert dedupe.body_from_arguments(["--fill"], str(tmp_path)) is None
|
|
@@ -13,6 +13,9 @@ if str(HOOKS_DIRECTORY) not in sys.path:
|
|
|
13
13
|
from blocking import followup_pr_dedupe as gate_dedupe
|
|
14
14
|
from blocking import pr_lifecycle_skill_gate as gate
|
|
15
15
|
from hooks_constants.pr_lifecycle_skill_gate_constants import DENY_REASON
|
|
16
|
+
from hooks_constants.pull_request_proof_constants import MISSING_PROOF_REASON
|
|
17
|
+
|
|
18
|
+
PROVEN_FOLLOWUP_BODY = "Follow-up to #1731\n\n## Proof in practice\nRan `python probe.py`.\n"
|
|
16
19
|
|
|
17
20
|
|
|
18
21
|
def _transcript(tmp_path: Path, skill_name: str | None = None, compact: bool = False) -> Path:
|
|
@@ -61,7 +64,7 @@ def test_unloaded_pull_request_command_is_denied(tmp_path: Path) -> None:
|
|
|
61
64
|
|
|
62
65
|
def test_loaded_skill_allows_action_silently(tmp_path: Path) -> None:
|
|
63
66
|
path = _transcript(tmp_path, "plugin:pr-lifecycle")
|
|
64
|
-
assert _run_main(_payload("gh pr
|
|
67
|
+
assert _run_main(_payload("gh pr view", path)) == (0, "")
|
|
65
68
|
|
|
66
69
|
|
|
67
70
|
def test_user_slash_command_loads_skill(tmp_path: Path) -> None:
|
|
@@ -127,7 +130,7 @@ def test_agent_id_with_path_separator_is_not_followed(tmp_path: Path) -> None:
|
|
|
127
130
|
|
|
128
131
|
|
|
129
132
|
def test_unreadable_or_missing_transcript_allows_silently(tmp_path: Path) -> None:
|
|
130
|
-
payload = _payload("gh pr
|
|
133
|
+
payload = _payload("gh pr view", tmp_path / "missing.jsonl")
|
|
131
134
|
assert _run_main(payload) == (0, "")
|
|
132
135
|
payload.pop("transcript_path")
|
|
133
136
|
assert _run_main(payload) == (0, "")
|
|
@@ -237,7 +240,7 @@ def test_hook_has_its_own_pre_tool_use_registration() -> None:
|
|
|
237
240
|
def test_loaded_skill_still_denies_a_second_followup_for_one_parent(tmp_path: Path) -> None:
|
|
238
241
|
payload = {
|
|
239
242
|
"tool_name": "mcp__github__create_pull_request",
|
|
240
|
-
"tool_input": {"owner": "jl-cmd", "repo": "claude-dev-env", "body":
|
|
243
|
+
"tool_input": {"owner": "jl-cmd", "repo": "claude-dev-env", "body": PROVEN_FOLLOWUP_BODY},
|
|
241
244
|
"transcript_path": str(_transcript(tmp_path, "pr-lifecycle")),
|
|
242
245
|
}
|
|
243
246
|
open_followup = {"number": 1769, "html_url": "https://github.com/jl-cmd/claude-dev-env/pull/1769", "body": "Follow-up to #1731"}
|
|
@@ -251,8 +254,14 @@ def test_loaded_skill_still_denies_a_second_followup_for_one_parent(tmp_path: Pa
|
|
|
251
254
|
def test_loaded_skill_allows_a_followup_when_the_read_fails(tmp_path: Path) -> None:
|
|
252
255
|
payload = {
|
|
253
256
|
"tool_name": "mcp__github__create_pull_request",
|
|
254
|
-
"tool_input": {"owner": "jl-cmd", "repo": "claude-dev-env", "body":
|
|
257
|
+
"tool_input": {"owner": "jl-cmd", "repo": "claude-dev-env", "body": PROVEN_FOLLOWUP_BODY},
|
|
255
258
|
"transcript_path": str(_transcript(tmp_path, "pr-lifecycle")),
|
|
256
259
|
}
|
|
257
260
|
with patch.object(gate_dedupe, "read_open_pull_requests", side_effect=OSError("offline")):
|
|
258
261
|
assert gate.decision_for(payload) is None
|
|
262
|
+
|
|
263
|
+
|
|
264
|
+
def test_loaded_skill_still_denies_a_new_pull_request_without_proof(tmp_path: Path) -> None:
|
|
265
|
+
exit_code, output = _run_main(_payload("gh pr create --body 'Adds a gate.'", _transcript(tmp_path, "pr-lifecycle")))
|
|
266
|
+
assert exit_code == 0
|
|
267
|
+
assert json.loads(output)["hookSpecificOutput"]["permissionDecisionReason"] == MISSING_PROOF_REASON
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
"""Behavior tests for the proof-in-practice check on a new pull request."""
|
|
2
|
+
|
|
3
|
+
import sys
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
|
|
6
|
+
HOOKS_DIRECTORY = Path(__file__).resolve().parent.parent
|
|
7
|
+
if str(HOOKS_DIRECTORY) not in sys.path:
|
|
8
|
+
sys.path.insert(0, str(HOOKS_DIRECTORY))
|
|
9
|
+
|
|
10
|
+
from blocking import pull_request_proof as gate
|
|
11
|
+
from hooks_constants.pull_request_proof_constants import MISSING_BODY_REASON, MISSING_PROOF_REASON
|
|
12
|
+
|
|
13
|
+
PROVEN_BODY = "## Summary\nAdds a gate.\n\n## Proof in practice\nRan `python run.py`.\n> denied\n"
|
|
14
|
+
UNPROVEN_BODY = "## Summary\nAdds a gate.\n\n## Verification\nUnit tests pass.\n"
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def _shell(command: str, working_directory: Path | None = None) -> dict[str, object]:
|
|
18
|
+
return {
|
|
19
|
+
"tool_name": "Bash",
|
|
20
|
+
"tool_input": {"command": command},
|
|
21
|
+
"cwd": str(working_directory or ""),
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def _mcp_create(body: object) -> dict[str, object]:
|
|
26
|
+
return {
|
|
27
|
+
"tool_name": "mcp__github__create_pull_request",
|
|
28
|
+
"tool_input": {"owner": "o", "repo": "r", "title": "t", "body": body},
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
def test_proof_section_should_stop_at_a_heading_of_the_same_level() -> None:
|
|
33
|
+
body = "### Proof in practice\nran `a`\n#### Child\nkept\n### Local checks\ndropped"
|
|
34
|
+
assert gate.proof_section(body) == "ran `a`\n#### Child\nkept"
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
def test_proof_section_should_return_none_without_the_heading() -> None:
|
|
38
|
+
assert gate.proof_section(UNPROVEN_BODY) is None
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def test_mcp_create_without_proof_should_be_denied() -> None:
|
|
42
|
+
assert gate.missing_proof_reason(_mcp_create(UNPROVEN_BODY)) == MISSING_PROOF_REASON
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
def test_mcp_create_with_no_body_should_be_denied() -> None:
|
|
46
|
+
assert gate.missing_proof_reason(_mcp_create(None)) == MISSING_PROOF_REASON
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def test_mcp_create_with_proof_should_pass() -> None:
|
|
50
|
+
assert gate.missing_proof_reason(_mcp_create(PROVEN_BODY)) is None
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def test_heading_with_no_command_should_be_denied() -> None:
|
|
54
|
+
body = "## Proof in practice\nIt works.\n"
|
|
55
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == MISSING_PROOF_REASON
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
def test_gh_create_body_file_with_proof_should_pass(tmp_path: Path) -> None:
|
|
59
|
+
(tmp_path / "body.md").write_text(PROVEN_BODY, encoding="utf-8")
|
|
60
|
+
payload = _shell("gh pr create --draft --body-file body.md", tmp_path)
|
|
61
|
+
assert gate.missing_proof_reason(payload) is None
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def test_gh_create_body_file_without_proof_should_be_denied(tmp_path: Path) -> None:
|
|
65
|
+
(tmp_path / "body.md").write_text(UNPROVEN_BODY, encoding="utf-8")
|
|
66
|
+
payload = _shell("gh pr create -F body.md", tmp_path)
|
|
67
|
+
assert gate.missing_proof_reason(payload) == MISSING_PROOF_REASON
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def test_gh_create_with_fill_should_ask_for_a_body() -> None:
|
|
71
|
+
assert gate.missing_proof_reason(_shell("gh pr create --fill")) == MISSING_BODY_REASON
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
def test_wrapped_gh_create_should_be_read() -> None:
|
|
75
|
+
payload = _shell("bash -c 'gh pr create --body \"no proof\"'")
|
|
76
|
+
assert gate.missing_proof_reason(payload) == MISSING_PROOF_REASON
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
def test_pull_request_script_create_should_be_read(tmp_path: Path) -> None:
|
|
80
|
+
(tmp_path / "body.md").write_text(UNPROVEN_BODY, encoding="utf-8")
|
|
81
|
+
payload = _shell("python3 scripts/pull_request.py create --body-file body.md", tmp_path)
|
|
82
|
+
assert gate.missing_proof_reason(payload) == MISSING_PROOF_REASON
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def test_other_pull_request_commands_should_pass() -> None:
|
|
86
|
+
for each_command in ("gh pr edit 3 --body x", "git push -u origin b", "gh pr view"):
|
|
87
|
+
assert gate.missing_proof_reason(_shell(each_command)) is None
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
def test_other_tools_should_pass() -> None:
|
|
91
|
+
payload = {"tool_name": "mcp__github__update_pull_request", "tool_input": {"body": "x"}}
|
|
92
|
+
assert gate.missing_proof_reason(payload) is None
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
"""Constants for the proof-in-practice check on a new pull request."""
|
|
2
|
+
|
|
3
|
+
PROOF_HEADING_PATTERN = r"^(#{1,6})[ \t]+proof in practice[ \t]*#*[ \t]*$"
|
|
4
|
+
NEXT_HEADING_PATTERN = r"^(#{1,6})[ \t]+\S"
|
|
5
|
+
COMMAND_MARKER = "`"
|
|
6
|
+
LINE_SEPARATOR = "\n"
|
|
7
|
+
ALL_GH_CREATE_WORDS = ["pr", "create"]
|
|
8
|
+
PULL_REQUEST_SCRIPT_NAME = "pull_request.py"
|
|
9
|
+
PULL_REQUEST_SCRIPT_CREATE_WORD = "create"
|
|
10
|
+
ALL_PYTHON_PROGRAM_NAMES = frozenset({"python", "python3", "py"})
|
|
11
|
+
CREATE_PULL_REQUEST_TOOL_SUFFIX = "__create_pull_request"
|
|
12
|
+
PROOF_GUIDE_PATH = "~/.claude/docs/rule-guides/proof-before-pull-request.md"
|
|
13
|
+
MISSING_BODY_REASON = (
|
|
14
|
+
"Pass the pull request body with --body or --body-file so this gate can read its"
|
|
15
|
+
" 'Proof in practice' section. Read " + PROOF_GUIDE_PATH + " for what counts as proof."
|
|
16
|
+
)
|
|
17
|
+
MISSING_PROOF_REASON = (
|
|
18
|
+
"This pull request body has no 'Proof in practice' section with the commands you ran."
|
|
19
|
+
" Before you open a pull request, run the changed thing the way its user runs it,"
|
|
20
|
+
" run the same scenario without the change, and quote the output lines that show"
|
|
21
|
+
" the difference. For a hook, skill, rule, setting, prompt or output style, spawn"
|
|
22
|
+
" each test session through the account broker. Then add a 'Proof in practice'"
|
|
23
|
+
" heading to the body that lists each command in backticks, quotes the output, states"
|
|
24
|
+
" the difference between the runs, and names anything still unproven. When you cannot"
|
|
25
|
+
" prove the change, leave the pull request unopened and tell the user what you could"
|
|
26
|
+
" not run. Read " + PROOF_GUIDE_PATH + " for the full rule."
|
|
27
|
+
)
|
package/package.json
CHANGED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
# Proof before a pull request
|
|
2
|
+
|
|
3
|
+
**When:** Before you open a pull request.
|
|
4
|
+
|
|
5
|
+
Run the change where it runs, with and without it, and quote both results under "Proof in practice" in the body. A hook denies a body without one. Unproven: leave it unopened and tell the user.
|
|
6
|
+
|
|
7
|
+
**Full text:** [`docs/rule-guides/proof-before-pull-request.md`](../docs/rule-guides/proof-before-pull-request.md).
|
|
@@ -1,12 +1,9 @@
|
|
|
1
1
|
# Present questions clearly
|
|
2
2
|
|
|
3
|
-
Before asking
|
|
3
|
+
Before asking, give a short formatted chat brief with context and each choice's effect. Add helpful visuals or artifacts.
|
|
4
4
|
|
|
5
|
-
For Codex
|
|
5
|
+
For Codex visual comparisons, find Page creation, visualization upload, and Page opening tools. Create the comparison, upload its visualization, and open its Page here before asking. If tools are missing, steps fail, or opening queues, compare in chat before asking. Put inline Visualize references only in final replies. An async acknowledgement or saved visual is context only; wait for and use the submitted answer before dependent work.
|
|
6
6
|
|
|
7
|
-
Ask one short, self-contained question with two or three
|
|
7
|
+
Ask one short, self-contained question with two or three choices. Give each a short label and one short sentence. Reuse labels in the brief, comparison, and picker.
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
Codex may expose `request_user_input_async` or `request_user_input`. Follow the tool's mode restrictions.
|
|
11
|
-
Put the question in its declared title or question field and choices in its declared string or object fields.
|
|
12
|
-
When no native tool is usable, ask the short question and choices in chat.
|
|
9
|
+
Find the native question tool and schema. Claude may use `AskUserQuestion`; Codex may use `request_user_input_async` or `request_user_input`. Honor mode rules. Put the question in its title or question field and choices in string or object fields. If unusable, ask in chat.
|