claude-dev-env 8.109.0 → 8.109.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/rule-guides/proof-before-pull-request.md +1 -1
- package/hooks/blocking/pr_lifecycle_skill_gate.py +5 -1
- package/hooks/blocking/pull_request_proof.py +125 -20
- package/hooks/blocking/test_pr_lifecycle_skill_gate.py +18 -1
- package/hooks/blocking/test_pull_request_proof.py +156 -0
- package/hooks/hooks_constants/pr_lifecycle_skill_gate_constants.py +2 -1
- package/hooks/hooks_constants/pull_request_proof_constants.py +21 -3
- package/package.json +1 -1
|
@@ -36,7 +36,7 @@ When you cannot prove it, leave the pull request unopened. Tell the owner in thr
|
|
|
36
36
|
|
|
37
37
|
## Enforcement
|
|
38
38
|
|
|
39
|
-
`hooks/blocking/pull_request_proof.py` reads the body of each new pull request before it opens. `pr_lifecycle_skill_gate.py` runs it on `gh pr create`, the `pull_request.py create` script, and each `mcp__*__create_pull_request` tool. The gate denies the call when the body has no "Proof in practice" heading, or when the text under that heading names no command in backticks. The gate also denies a section whose only commands run a test runner, such as `pytest`, `python -m unittest`, `uv run pytest`, `npm test`, `npx vitest`, `jest`, `go test` or `cargo test`. Quotes around a runner word, interpreter flags before `-m`, and `coverage run -m unittest` are still test runs. A command is an inline backticked span or a `$ ` line inside a fenced block. A span that starts with a path and line, such as `gate.py:12`, is not a command. A span that holds one path or file name, such as `src/x.py`, is not a command unless that span runs a test runner, such as `py.test` or `pytest.exe`. A test run piped into `tee`, `tail`, `head`, `grep` or `cat` is still a test run. The deny quotes the line "Unit tests are welcome as extra evidence. The live run is the proof." and asks for a run of the changed thing the way its user runs it, without the change and with it. The gate also denies a section whose prose says part of the change is not proven, not tested or not verified, with "yet", "been" or "fully" allowed after the "not", or says unproven, untested or unverified. The deny quotes that line, and asks for the run before the pull request opens, or for the line under a "Not covered" heading when the change does not target it. Words inside backticks or a fenced block are quoted output and do not count. A section with one command that is not a test run, and no such line, passes. A `gh pr create` with no `--body` or `--body-file` is denied with a request for the body.
|
|
39
|
+
`hooks/blocking/pull_request_proof.py` reads the body of each new pull request before it opens. `pr_lifecycle_skill_gate.py` runs it on `gh pr create`, the `pull_request.py create` script, and each `mcp__*__create_pull_request` tool. The gate denies the call when the body has no "Proof in practice" heading, or when the text under that heading names no command in backticks. The gate also denies a section whose only commands run a test runner, such as `pytest`, `python -m unittest`, `uv run pytest`, `npm test`, `npx vitest`, `jest`, `go test` or `cargo test`. Quotes around a runner word, interpreter flags before `-m`, and `coverage run -m unittest` are still test runs. A command is an inline backticked span or a `$ ` line inside a fenced block. A span that starts with a path and line, such as `gate.py:12`, is not a command. A span that holds one path or file name, such as `src/x.py`, is not a command unless that span runs a test runner, such as `py.test` or `pytest.exe`. A test run piped into `tee`, `tail`, `head`, `grep` or `cat` is still a test run. The deny quotes the line "Unit tests are welcome as extra evidence. The live run is the proof." and asks for a run of the changed thing the way its user runs it, without the change and with it. The gate also denies a section whose prose says part of the change is not proven, not tested or not verified, with "yet", "been" or "fully" allowed after the "not", or says unproven, untested or unverified. The deny quotes that line, and asks for the run before the pull request opens, or for the line under a "Not covered" heading when the change does not target it. The same denial covers a proof line that opens with "not run", "not exercised" or "not executed" (a "yet" or "been" may follow the "not") or says "has not been run". A proof line with a merge word (merge, merged, merges or merging, so post-merge too) is denied with its own reason: run the check on the branch, and put a file or command name that contains merge in backticks. A step that needs the merged code to exist goes under a "Not covered" heading at the same level as the proof heading, as a line that opens with "not run". `UNPROVEN_ADMISSION_PATTERN` and `MERGE_WORD_PATTERN` list the shapes. Words inside backticks or a fenced block are quoted output and do not count. A section with one command that is not a test run, and no such line, passes. A `gh pr create` with no `--body` or `--body-file` is denied with a request for the body. A body edit through `update_pull_request`, `gh pr edit` or the `pull_request.py edit` script gets the same checks on its proof section that a new pull request gets, so an edit cannot swap in a proof that a new pull request would be denied. An edit with no proof section passes. A body the gate cannot read passes, such as an inline multi-line `--body`, a body from standard input, or a body written through `gh api`. Merges and pushes pass.
|
|
40
40
|
|
|
41
41
|
When the checked-out branch changes a file people see (`.html`, `.htm`, `.css`, `.scss`, `.tsx`, `.jsx`, `.vue`, `.svelte`), the gate also denies a body whose proof section holds no picture: a markdown image, an image URL, or an Artifact page link that shows the screenshots. Render the change, read the screenshot, and put it in the section, so the reader sees what you looked at. When git in the session directory cannot list the branch's changed files, the gate asks for the picture as well.
|
|
42
42
|
|
|
@@ -67,6 +67,7 @@ from hooks_constants.pre_tool_use_stdin import print_decision_from_stdin
|
|
|
67
67
|
from hooks_constants.setup_project_paths_constants import DECODE_ERRORS_POLICY, UTF8_ENCODING
|
|
68
68
|
from blocking.followup_pr_dedupe import create_tool_fields, duplicate_followup_reason
|
|
69
69
|
from blocking.pull_request_proof import (
|
|
70
|
+
edit_proof_reason,
|
|
70
71
|
is_feature_pull_request,
|
|
71
72
|
missing_eval_section_reason,
|
|
72
73
|
missing_existing_work_reason,
|
|
@@ -331,6 +332,9 @@ def decision_for(all_payload_fields: dict[str, object]) -> dict[str, object] | N
|
|
|
331
332
|
|
|
332
333
|
pr-lifecycle skill not loaded -> DENY_REASON
|
|
333
334
|
new pull request with no proof section -> MISSING_PROOF_REASON
|
|
335
|
+
proof, or a body edit, says not run -> UNPROVEN_PROOF_REASON
|
|
336
|
+
proof, or a body edit, holds a merge word -> DEFERRED_PROOF_REASON
|
|
337
|
+
body edit whose proof has no command or only tests -> MISSING_PROOF_REASON or TEST_ONLY_PROOF_REASON
|
|
334
338
|
new pull request with no existing-work section -> MISSING_EXISTING_WORK_REASON
|
|
335
339
|
feature pull request with no eval section -> MISSING_EVAL_SECTION_REASON
|
|
336
340
|
changed page whose proof shows no picture -> MISSING_LOOK_REASON
|
|
@@ -344,7 +348,7 @@ def decision_for(all_payload_fields: dict[str, object]) -> dict[str, object] | N
|
|
|
344
348
|
return None
|
|
345
349
|
if _is_skill_unloaded(all_payload_fields):
|
|
346
350
|
return deny_decision(DENY_REASON)
|
|
347
|
-
proof_reason = missing_proof_reason(all_payload_fields)
|
|
351
|
+
proof_reason = missing_proof_reason(all_payload_fields) or edit_proof_reason(all_payload_fields)
|
|
348
352
|
if proof_reason is not None:
|
|
349
353
|
return deny_decision(proof_reason)
|
|
350
354
|
search_reason = missing_existing_work_reason(all_payload_fields)
|
|
@@ -14,13 +14,16 @@ if hooks_directory not in sys.path:
|
|
|
14
14
|
sys.path.insert(0, hooks_directory)
|
|
15
15
|
|
|
16
16
|
from blocking.followup_pr_dedupe import body_from_arguments, create_tool_fields, title_from_arguments
|
|
17
|
-
from hooks_constants.pr_lifecycle_skill_gate_constants import SHELL_TOOL_NAMES
|
|
17
|
+
from hooks_constants.pr_lifecycle_skill_gate_constants import SHELL_TOOL_NAMES, UPDATE_PULL_REQUEST_TOOL_SUFFIX
|
|
18
18
|
from hooks_constants.pull_request_proof_constants import (
|
|
19
19
|
ALL_GH_CREATE_WORDS,
|
|
20
|
+
ALL_GH_EDIT_WORDS,
|
|
20
21
|
ALL_PYTHON_PROGRAM_NAMES,
|
|
21
22
|
ALL_VISIBLE_FILE_SUFFIXES,
|
|
22
23
|
CHANGED_FILES_SEPARATOR,
|
|
23
24
|
COMMAND_MARKER,
|
|
25
|
+
DEFERRED_PROOF_REASON,
|
|
26
|
+
MERGE_WORD_PATTERN,
|
|
24
27
|
ALL_BASE_FLAGS,
|
|
25
28
|
ALL_DEFAULT_BRANCH_REFERENCES,
|
|
26
29
|
ALL_HEAD_FLAGS,
|
|
@@ -50,6 +53,7 @@ from hooks_constants.pull_request_proof_constants import (
|
|
|
50
53
|
PROOF_HEADING_PATTERN,
|
|
51
54
|
PROOF_IMAGE_PATTERN,
|
|
52
55
|
PULL_REQUEST_SCRIPT_CREATE_WORD,
|
|
56
|
+
PULL_REQUEST_SCRIPT_EDIT_WORD,
|
|
53
57
|
PULL_REQUEST_SCRIPT_NAME,
|
|
54
58
|
ALL_TEST_RUNNER_COMMAND_PREFIXES,
|
|
55
59
|
CODE_FENCE_MARKER,
|
|
@@ -148,18 +152,20 @@ def _script_arguments(program: str, all_arguments: list[str]) -> list[str] | Non
|
|
|
148
152
|
return all_arguments[1:] if script_name == PULL_REQUEST_SCRIPT_NAME else None
|
|
149
153
|
|
|
150
154
|
|
|
151
|
-
def
|
|
155
|
+
def _script_word_arguments(program: str, all_arguments: list[str], word: str) -> list[str] | None:
|
|
152
156
|
all_script_arguments = _script_arguments(program, all_arguments)
|
|
153
|
-
if all_script_arguments is None or all_script_arguments[:1] != [
|
|
157
|
+
if all_script_arguments is None or all_script_arguments[:1] != [word]:
|
|
154
158
|
return None
|
|
155
159
|
return all_script_arguments[1:]
|
|
156
160
|
|
|
157
161
|
|
|
158
|
-
def
|
|
162
|
+
def _command_arguments(
|
|
163
|
+
all_segment_tokens: list[str], all_gh_words: list[str], script_word: str
|
|
164
|
+
) -> list[str] | None:
|
|
159
165
|
program, all_arguments = segment_program_and_arguments(all_segment_tokens)
|
|
160
|
-
if program == "gh" and all_arguments[: len(
|
|
161
|
-
return all_arguments[len(
|
|
162
|
-
return
|
|
166
|
+
if program == "gh" and all_arguments[: len(all_gh_words)] == all_gh_words:
|
|
167
|
+
return all_arguments[len(all_gh_words) :]
|
|
168
|
+
return _script_word_arguments(program, all_arguments, script_word)
|
|
163
169
|
|
|
164
170
|
|
|
165
171
|
def _shell_command(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
@@ -170,15 +176,23 @@ def _shell_command(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
|
170
176
|
return command
|
|
171
177
|
|
|
172
178
|
|
|
173
|
-
def
|
|
179
|
+
def _all_argument_lists(command: str, all_gh_words: list[str], script_word: str) -> Iterator[list[str]]:
|
|
174
180
|
return (
|
|
175
|
-
|
|
181
|
+
all_arguments
|
|
176
182
|
for each_text in all_wrapped_command_texts(command)
|
|
177
183
|
for each_segment, _each_following_operator in pipeline_segments_for_command(each_text)
|
|
178
|
-
if (
|
|
184
|
+
if (all_arguments := _command_arguments(each_segment, all_gh_words, script_word)) is not None
|
|
179
185
|
)
|
|
180
186
|
|
|
181
187
|
|
|
188
|
+
def _all_create_argument_lists(command: str) -> Iterator[list[str]]:
|
|
189
|
+
return _all_argument_lists(command, ALL_GH_CREATE_WORDS, PULL_REQUEST_SCRIPT_CREATE_WORD)
|
|
190
|
+
|
|
191
|
+
|
|
192
|
+
def _all_edit_argument_lists(command: str) -> Iterator[list[str]]:
|
|
193
|
+
return _all_argument_lists(command, ALL_GH_EDIT_WORDS, PULL_REQUEST_SCRIPT_EDIT_WORD)
|
|
194
|
+
|
|
195
|
+
|
|
182
196
|
def _shell_create_calls(command: str, working_directory: str) -> list[CreateCall]:
|
|
183
197
|
return [
|
|
184
198
|
CreateCall(
|
|
@@ -443,15 +457,46 @@ def unproven_admission(section: str) -> str | None:
|
|
|
443
457
|
|
|
444
458
|
The markers are not proven, not tested and not verified, each with any
|
|
445
459
|
run of yet, been or fully after the not, plus unproven, untested and
|
|
446
|
-
unverified
|
|
447
|
-
|
|
460
|
+
unverified. A line or list item that opens with "not run", "not ran",
|
|
461
|
+
"not exercised" or "not executed", each with any run of yet or been after
|
|
462
|
+
the not, is one too, and so is "has not been run", "have not been exercised"
|
|
463
|
+
or "hasn't been executed", with "yet" allowed before "been". Case is
|
|
464
|
+
ignored.
|
|
465
|
+
Inline code spans and fenced blocks are skipped, so output quoted there
|
|
466
|
+
never counts.
|
|
448
467
|
"""
|
|
468
|
+
return _first_prose_match(section, UNPROVEN_ADMISSION_PATTERN)
|
|
469
|
+
|
|
470
|
+
|
|
471
|
+
def _first_prose_match(section: str, pattern: str) -> str | None:
|
|
449
472
|
for each_line, each_prose in _prose_lines(section):
|
|
450
|
-
if re.search(
|
|
473
|
+
if re.search(pattern, each_prose, re.IGNORECASE):
|
|
451
474
|
return each_line.strip()
|
|
452
475
|
return None
|
|
453
476
|
|
|
454
477
|
|
|
478
|
+
def merge_word_line(section: str) -> str | None:
|
|
479
|
+
"""Return the first prose line of a proof section that holds a merge word, or None.
|
|
480
|
+
|
|
481
|
+
::
|
|
482
|
+
|
|
483
|
+
"I check the publish after the merge." -> that line
|
|
484
|
+
"A session verifies it after merging." -> that line
|
|
485
|
+
"Post-merge, a session confirms it." -> that line
|
|
486
|
+
"Ran `git merge origin/main`, then `python run.py`." -> None
|
|
487
|
+
"Verified on the branch." -> None
|
|
488
|
+
|
|
489
|
+
The words are merge, merged, merges and merging, matched without regard to
|
|
490
|
+
case, so post-merge counts. Inline code spans and fenced blocks are skipped,
|
|
491
|
+
so a command or file name that holds the word passes when it sits in
|
|
492
|
+
backticks.
|
|
493
|
+
|
|
494
|
+
Args:
|
|
495
|
+
section: The text under the 'Proof in practice' heading.
|
|
496
|
+
"""
|
|
497
|
+
return _first_prose_match(section, MERGE_WORD_PATTERN)
|
|
498
|
+
|
|
499
|
+
|
|
455
500
|
def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
456
501
|
"""Return a deny reason when a new pull request's body has no proof, else None.
|
|
457
502
|
|
|
@@ -464,6 +509,8 @@ def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None
|
|
|
464
509
|
proof whose only command is `python -q -m unittest` -> TEST_ONLY_PROOF_REASON
|
|
465
510
|
proof with `pytest` and `python run.py` -> None
|
|
466
511
|
proof whose prose says "Not proven here: the run" -> UNPROVEN_PROOF_REASON
|
|
512
|
+
proof whose prose says "Not run: the live publish" -> UNPROVEN_PROOF_REASON
|
|
513
|
+
proof prose that says "I verify it after the merge" -> DEFERRED_PROOF_REASON
|
|
467
514
|
gh pr create --fill -> MISSING_BODY_REASON
|
|
468
515
|
gh pr edit, git push, any other tool -> None
|
|
469
516
|
|
|
@@ -479,7 +526,8 @@ def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None
|
|
|
479
526
|
A pipe into tee, tail, head, grep or cat keeps a test run a test run.
|
|
480
527
|
A section whose prose says part of the change is not proven, untested or
|
|
481
528
|
unverified is denied, with the line quoted; unproven_admission lists the
|
|
482
|
-
markers.
|
|
529
|
+
markers. A prose line with a merge word is denied in the proof section;
|
|
530
|
+
merge_word_line lists the words.
|
|
483
531
|
|
|
484
532
|
Args:
|
|
485
533
|
all_payload_fields: The parsed PreToolUse input.
|
|
@@ -488,16 +536,73 @@ def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None
|
|
|
488
536
|
if each_body is None:
|
|
489
537
|
return MISSING_BODY_REASON
|
|
490
538
|
section = proof_section(each_body)
|
|
491
|
-
if not section
|
|
539
|
+
if not section:
|
|
492
540
|
return MISSING_PROOF_REASON
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
if admission is not None:
|
|
497
|
-
return UNPROVEN_PROOF_REASON.format(admission=admission)
|
|
541
|
+
section_reason = _proof_section_reason(section)
|
|
542
|
+
if section_reason is not None:
|
|
543
|
+
return section_reason
|
|
498
544
|
return None
|
|
499
545
|
|
|
500
546
|
|
|
547
|
+
def _proof_section_reason(section: str) -> str | None:
|
|
548
|
+
if COMMAND_MARKER not in section:
|
|
549
|
+
return MISSING_PROOF_REASON
|
|
550
|
+
if _proof_runs_only_tests(section):
|
|
551
|
+
return TEST_ONLY_PROOF_REASON
|
|
552
|
+
admission = unproven_admission(section)
|
|
553
|
+
if admission is not None:
|
|
554
|
+
return UNPROVEN_PROOF_REASON.format(admission=admission)
|
|
555
|
+
deferral = merge_word_line(section)
|
|
556
|
+
return None if deferral is None else DEFERRED_PROOF_REASON.format(deferral=deferral)
|
|
557
|
+
|
|
558
|
+
|
|
559
|
+
def edit_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
560
|
+
"""Return a deny reason when a pull request body edit weakens its proof section, else None.
|
|
561
|
+
|
|
562
|
+
::
|
|
563
|
+
|
|
564
|
+
update_pull_request, proof section says "Not run: x" -> UNPROVEN_PROOF_REASON
|
|
565
|
+
gh pr edit --body-file, proof prose says "after the merge" -> DEFERRED_PROOF_REASON
|
|
566
|
+
update_pull_request, proof whose only command is `pytest` -> TEST_ONLY_PROOF_REASON
|
|
567
|
+
update_pull_request, body with no proof section -> None
|
|
568
|
+
update_pull_request with no body, create calls, any tool -> None
|
|
569
|
+
|
|
570
|
+
An edit with no proof heading passes. A proof heading that is there meets
|
|
571
|
+
the rules a new pull request meets: a command in backticks, more than test
|
|
572
|
+
runs, no unrun line and no merge word. The edit comes through the
|
|
573
|
+
update_pull_request tool, gh pr edit, or the pull_request.py edit script.
|
|
574
|
+
A body this gate cannot read passes unread: an inline multi-line --body,
|
|
575
|
+
--body-file - from standard input, a $(...) body, a --body-file path that
|
|
576
|
+
an earlier cd changes, a gh global option before pr edit, or a gh api call.
|
|
577
|
+
|
|
578
|
+
Args:
|
|
579
|
+
all_payload_fields: The parsed PreToolUse input.
|
|
580
|
+
"""
|
|
581
|
+
for each_body in _edit_bodies(all_payload_fields):
|
|
582
|
+
section = proof_section(each_body)
|
|
583
|
+
edit_reason = None if section is None else _proof_section_reason(section)
|
|
584
|
+
if edit_reason is not None:
|
|
585
|
+
return edit_reason
|
|
586
|
+
return None
|
|
587
|
+
|
|
588
|
+
|
|
589
|
+
def _edit_bodies(all_payload_fields: Mapping[str, object]) -> list[str]:
|
|
590
|
+
tool_name = all_payload_fields.get("tool_name")
|
|
591
|
+
tool_input = all_payload_fields.get("tool_input")
|
|
592
|
+
if isinstance(tool_name, str) and tool_name.endswith(UPDATE_PULL_REQUEST_TOOL_SUFFIX):
|
|
593
|
+
body = tool_input.get("body") if isinstance(tool_input, dict) else None
|
|
594
|
+
return [body] if isinstance(body, str) else []
|
|
595
|
+
command = _shell_command(all_payload_fields)
|
|
596
|
+
if command is None:
|
|
597
|
+
return []
|
|
598
|
+
working_directory = all_payload_fields.get("cwd")
|
|
599
|
+
all_bodies = [
|
|
600
|
+
body_from_arguments(each_arguments, working_directory if isinstance(working_directory, str) else "")
|
|
601
|
+
for each_arguments in _all_edit_argument_lists(command)
|
|
602
|
+
]
|
|
603
|
+
return [each_body for each_body in all_bodies if each_body is not None]
|
|
604
|
+
|
|
605
|
+
|
|
501
606
|
def missing_existing_work_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
502
607
|
"""Return a deny reason when a new pull request's body has no existing-work section, else None.
|
|
503
608
|
|
|
@@ -14,7 +14,11 @@ if str(HOOKS_DIRECTORY) not in sys.path:
|
|
|
14
14
|
from blocking import followup_pr_dedupe as gate_dedupe
|
|
15
15
|
from blocking import pr_lifecycle_skill_gate as gate
|
|
16
16
|
from hooks_constants.pr_lifecycle_skill_gate_constants import DENY_REASON
|
|
17
|
-
from hooks_constants.pull_request_proof_constants import
|
|
17
|
+
from hooks_constants.pull_request_proof_constants import (
|
|
18
|
+
MISSING_BUILD_EVAL_REASON,
|
|
19
|
+
MISSING_PROOF_REASON,
|
|
20
|
+
UNPROVEN_PROOF_REASON,
|
|
21
|
+
)
|
|
18
22
|
|
|
19
23
|
PROVEN_FOLLOWUP_BODY = (
|
|
20
24
|
"Follow-up to #1731\n\n## Existing work\nNothing found in open or merged pull requests.\n\n"
|
|
@@ -352,6 +356,19 @@ def test_loaded_skill_still_denies_a_new_pull_request_without_proof(tmp_path: Pa
|
|
|
352
356
|
assert json.loads(output)["hookSpecificOutput"]["permissionDecisionReason"] == MISSING_PROOF_REASON
|
|
353
357
|
|
|
354
358
|
|
|
359
|
+
def test_loaded_skill_denies_a_body_edit_that_adds_an_unrun_proof_line(tmp_path: Path) -> None:
|
|
360
|
+
unrun_line = "Not run: the first publish on the live remote."
|
|
361
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{unrun_line}\n"
|
|
362
|
+
payload = {
|
|
363
|
+
"tool_name": "mcp__github__update_pull_request",
|
|
364
|
+
"tool_input": {"owner": "jl-cmd", "repo": "claude-dev-env", "pullNumber": 7, "body": body},
|
|
365
|
+
"transcript_path": str(_transcript(tmp_path, "pr-lifecycle")),
|
|
366
|
+
}
|
|
367
|
+
decision = gate.decision_for(payload)
|
|
368
|
+
assert decision is not None
|
|
369
|
+
assert decision["hookSpecificOutput"]["permissionDecisionReason"] == UNPROVEN_PROOF_REASON.format(admission=unrun_line)
|
|
370
|
+
|
|
371
|
+
|
|
355
372
|
EVALUATED_BODY = (
|
|
356
373
|
"## Existing work\nNothing found in open or merged pull requests.\n\n"
|
|
357
374
|
"## Proof in practice\nRan `python probe.py`.\n\n"
|
|
@@ -13,6 +13,7 @@ if str(HOOKS_DIRECTORY) not in sys.path:
|
|
|
13
13
|
from blocking import pull_request_proof as gate
|
|
14
14
|
from hooks_constants.pull_request_proof_constants import (
|
|
15
15
|
COMPARE_FILE_LIST_LIMIT,
|
|
16
|
+
DEFERRED_PROOF_REASON,
|
|
16
17
|
MISSING_BODY_REASON,
|
|
17
18
|
MISSING_EVAL_SECTION_REASON,
|
|
18
19
|
MISSING_EXISTING_WORK_REASON,
|
|
@@ -686,3 +687,158 @@ def test_is_create_call_should_name_shell_and_mcp_creates() -> None:
|
|
|
686
687
|
assert gate.is_create_call(_mcp_create("b"))
|
|
687
688
|
assert not gate.is_create_call(_shell("gh pr edit 7 --body b"))
|
|
688
689
|
assert not gate.is_create_call({"tool_name": "Read", "tool_input": {"file_path": "a.py"}})
|
|
690
|
+
|
|
691
|
+
|
|
692
|
+
@pytest.mark.parametrize(
|
|
693
|
+
"admission",
|
|
694
|
+
[
|
|
695
|
+
"Not run: the first publish on the live remote.",
|
|
696
|
+
"- Not run: a push race between live sessions.",
|
|
697
|
+
"1. Not run: the live publish.",
|
|
698
|
+
"The live publish has not yet been run.",
|
|
699
|
+
"It hasn't been run on the target.",
|
|
700
|
+
"The dialer path has not been exercised here.",
|
|
701
|
+
],
|
|
702
|
+
)
|
|
703
|
+
def test_each_unrun_wording_in_proof_prose_should_be_denied(admission: str) -> None:
|
|
704
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{admission}\n"
|
|
705
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == UNPROVEN_PROOF_REASON.format(admission=admission)
|
|
706
|
+
|
|
707
|
+
|
|
708
|
+
@pytest.mark.parametrize(
|
|
709
|
+
"merge_line",
|
|
710
|
+
[
|
|
711
|
+
"I check the first live publish after the merge.",
|
|
712
|
+
"The seed step happens once merged.",
|
|
713
|
+
"Post-merge, a session confirms the publish.",
|
|
714
|
+
"A session verifies it after merging.",
|
|
715
|
+
"After it merges, I verify the publish.",
|
|
716
|
+
"I will verify the publish on merge.",
|
|
717
|
+
"I verify the live publish post merge.",
|
|
718
|
+
"Verified the output after merging main into the branch.",
|
|
719
|
+
],
|
|
720
|
+
)
|
|
721
|
+
def test_each_merge_word_in_proof_prose_should_be_denied(merge_line: str) -> None:
|
|
722
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{merge_line}\n"
|
|
723
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == DEFERRED_PROOF_REASON.format(deferral=merge_line)
|
|
724
|
+
|
|
725
|
+
|
|
726
|
+
@pytest.mark.parametrize(
|
|
727
|
+
"prose",
|
|
728
|
+
[
|
|
729
|
+
"The gate does not run on edits.",
|
|
730
|
+
"With the change, the hook is not run on edits.",
|
|
731
|
+
"Without the change, the hook was not run for edits.",
|
|
732
|
+
"Before this change the gate had not run on edits.",
|
|
733
|
+
"Ran `git merge origin/main`, then `python run.py`.",
|
|
734
|
+
"Confirmed the hold label stays until the PR lands.",
|
|
735
|
+
"Not run-dependent.",
|
|
736
|
+
"Without the change the publish failed; with it the publish passed.",
|
|
737
|
+
],
|
|
738
|
+
)
|
|
739
|
+
def test_proof_prose_without_an_unrun_line_or_a_merge_word_should_pass(prose: str) -> None:
|
|
740
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{prose}\n"
|
|
741
|
+
assert gate.missing_proof_reason(_mcp_create(body)) is None
|
|
742
|
+
|
|
743
|
+
|
|
744
|
+
def test_not_covered_heading_nested_under_the_proof_heading_should_stay_proof_text() -> None:
|
|
745
|
+
line = "Not run: an older shape."
|
|
746
|
+
body = f"## Proof in practice\nRan `python run.py`.\n\n### Not covered\n{line}\n"
|
|
747
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == UNPROVEN_PROOF_REASON.format(admission=line)
|
|
748
|
+
|
|
749
|
+
|
|
750
|
+
def test_proof_made_only_of_a_nested_not_covered_line_should_be_denied() -> None:
|
|
751
|
+
line = "Not run: `python publish.py`"
|
|
752
|
+
body = f"## Proof in practice\n### Not covered\n{line}\n"
|
|
753
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == UNPROVEN_PROOF_REASON.format(admission=line)
|
|
754
|
+
|
|
755
|
+
|
|
756
|
+
def test_merge_step_under_a_same_level_not_covered_heading_should_pass() -> None:
|
|
757
|
+
body = (
|
|
758
|
+
"## Proof in practice\nRan `python run.py`.\n\n## Not covered\n\n"
|
|
759
|
+
"Not run: the publish check that follows the merge.\n"
|
|
760
|
+
)
|
|
761
|
+
assert gate.missing_proof_reason(_mcp_create(body)) is None
|
|
762
|
+
|
|
763
|
+
|
|
764
|
+
def test_merge_word_line_should_return_the_line_or_none() -> None:
|
|
765
|
+
assert gate.merge_word_line("Ran `python run.py`.\nChecked after the merge.\n") == "Checked after the merge."
|
|
766
|
+
assert gate.merge_word_line("Ran `git merge main`.\n") is None
|
|
767
|
+
assert gate.merge_word_line("```\nmerged ok\n```\n") is None
|
|
768
|
+
|
|
769
|
+
|
|
770
|
+
def _mcp_update(body: object) -> dict[str, object]:
|
|
771
|
+
return {
|
|
772
|
+
"tool_name": "mcp__github__update_pull_request",
|
|
773
|
+
"tool_input": {"owner": "o", "repo": "r", "pullNumber": 7, "body": body},
|
|
774
|
+
}
|
|
775
|
+
|
|
776
|
+
|
|
777
|
+
def test_body_edit_that_adds_an_unrun_proof_line_should_be_denied() -> None:
|
|
778
|
+
line = "Not run: the first publish on the live remote."
|
|
779
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{line}\n"
|
|
780
|
+
assert gate.edit_proof_reason(_mcp_update(body)) == UNPROVEN_PROOF_REASON.format(admission=line)
|
|
781
|
+
|
|
782
|
+
|
|
783
|
+
def test_body_edit_that_adds_a_merge_word_to_the_proof_should_be_denied() -> None:
|
|
784
|
+
line = "I check the first live publish after the merge."
|
|
785
|
+
body = f"## Proof in practice\nRan `python run.py`.\n{line}\n"
|
|
786
|
+
assert gate.edit_proof_reason(_mcp_update(body)) == DEFERRED_PROOF_REASON.format(deferral=line)
|
|
787
|
+
|
|
788
|
+
|
|
789
|
+
@pytest.mark.parametrize(
|
|
790
|
+
"command",
|
|
791
|
+
[
|
|
792
|
+
"gh pr edit 7 --body-file body.md",
|
|
793
|
+
"python pull_request.py edit 7 --body-file body.md",
|
|
794
|
+
],
|
|
795
|
+
)
|
|
796
|
+
def test_shell_body_edit_with_an_unrun_proof_line_should_be_denied(command: str, tmp_path: Path) -> None:
|
|
797
|
+
line = "Not run: the live publish."
|
|
798
|
+
(tmp_path / "body.md").write_text(f"## Proof in practice\nRan `python run.py`.\n{line}\n", encoding="utf-8")
|
|
799
|
+
assert gate.edit_proof_reason(_shell(command, tmp_path)) == UNPROVEN_PROOF_REASON.format(admission=line)
|
|
800
|
+
|
|
801
|
+
|
|
802
|
+
def test_body_edit_that_leaves_only_a_test_run_in_the_proof_should_be_denied() -> None:
|
|
803
|
+
body = "## Proof in practice\nRan `pytest -q`.\n"
|
|
804
|
+
assert gate.edit_proof_reason(_mcp_update(body)) == TEST_ONLY_PROOF_REASON
|
|
805
|
+
|
|
806
|
+
|
|
807
|
+
def test_body_edit_that_leaves_an_empty_proof_heading_should_be_denied() -> None:
|
|
808
|
+
body = "## Summary\nx\n\n## Proof in practice\n\n## Notes\nok\n"
|
|
809
|
+
assert gate.edit_proof_reason(_mcp_update(body)) == MISSING_PROOF_REASON
|
|
810
|
+
|
|
811
|
+
|
|
812
|
+
def test_body_edit_that_leaves_a_proof_section_with_no_command_should_be_denied() -> None:
|
|
813
|
+
body = "## Proof in practice\nIt works.\n"
|
|
814
|
+
assert gate.edit_proof_reason(_mcp_update(body)) == MISSING_PROOF_REASON
|
|
815
|
+
|
|
816
|
+
|
|
817
|
+
def test_gh_command_with_other_words_should_not_read_as_an_edit(tmp_path: Path) -> None:
|
|
818
|
+
(tmp_path / "body.md").write_text("## Proof in practice\nRan `python run.py`.\nNot run: x.\n", encoding="utf-8")
|
|
819
|
+
assert gate.edit_proof_reason(_shell("gh pr view edit --body-file body.md", tmp_path)) is None
|
|
820
|
+
|
|
821
|
+
|
|
822
|
+
def test_body_edit_without_a_proof_section_should_pass() -> None:
|
|
823
|
+
assert gate.edit_proof_reason(_mcp_update("## Summary\nNot run: nothing here is a proof.\n")) is None
|
|
824
|
+
|
|
825
|
+
|
|
826
|
+
def test_body_edit_with_a_clean_proof_should_pass() -> None:
|
|
827
|
+
assert gate.edit_proof_reason(_mcp_update(PROVEN_BODY)) is None
|
|
828
|
+
|
|
829
|
+
|
|
830
|
+
def test_body_edit_under_a_same_level_not_covered_heading_should_pass() -> None:
|
|
831
|
+
body = "## Proof in practice\nRan `python run.py`.\n\n## Not covered\n\nNot run: the check after the merge.\n"
|
|
832
|
+
assert gate.edit_proof_reason(_mcp_update(body)) is None
|
|
833
|
+
|
|
834
|
+
|
|
835
|
+
def test_edit_without_a_readable_body_should_pass(tmp_path: Path) -> None:
|
|
836
|
+
assert gate.edit_proof_reason(_mcp_update(None)) is None
|
|
837
|
+
assert gate.edit_proof_reason(_shell("gh pr edit 7 --title t", tmp_path)) is None
|
|
838
|
+
assert gate.edit_proof_reason(_shell("gh pr edit 7 --body-file -", tmp_path)) is None
|
|
839
|
+
|
|
840
|
+
|
|
841
|
+
def test_create_calls_and_other_tools_should_pass_the_edit_check(tmp_path: Path) -> None:
|
|
842
|
+
unrun_body = "## Proof in practice\nRan `python run.py`.\nNot run: x.\n"
|
|
843
|
+
assert gate.edit_proof_reason(_mcp_create(unrun_body)) is None
|
|
844
|
+
assert gate.edit_proof_reason(_shell("git push -u origin b", tmp_path)) is None
|
|
@@ -11,11 +11,12 @@ DENY_REASON = (
|
|
|
11
11
|
SKILL_NAME = "pr-lifecycle"
|
|
12
12
|
ALL_SLASH_COMMAND_MARKERS = ("<command-name>/pr-lifecycle</command-name>",)
|
|
13
13
|
SHELL_TOOL_NAMES = frozenset({"Bash", "PowerShell"})
|
|
14
|
+
UPDATE_PULL_REQUEST_TOOL_SUFFIX = "__update_pull_request"
|
|
14
15
|
ALL_GITHUB_MCP_TOOL_SUFFIXES = (
|
|
15
16
|
"__create_pull_request",
|
|
16
17
|
"__merge_pull_request",
|
|
17
18
|
"__enable_pr_auto_merge",
|
|
18
|
-
|
|
19
|
+
UPDATE_PULL_REQUEST_TOOL_SUFFIX,
|
|
19
20
|
)
|
|
20
21
|
ALL_TRANSCRIPT_PATH_FIELDS = ("transcript_path", "agent_transcript_path")
|
|
21
22
|
SESSION_TRANSCRIPT_PATH_FIELD = "transcript_path"
|
|
@@ -5,8 +5,10 @@ NEXT_HEADING_PATTERN = r"^(#{1,6})[ \t]+\S"
|
|
|
5
5
|
COMMAND_MARKER = "`"
|
|
6
6
|
LINE_SEPARATOR = "\n"
|
|
7
7
|
ALL_GH_CREATE_WORDS = ["pr", "create"]
|
|
8
|
+
ALL_GH_EDIT_WORDS = ["pr", "edit"]
|
|
8
9
|
PULL_REQUEST_SCRIPT_NAME = "pull_request.py"
|
|
9
10
|
PULL_REQUEST_SCRIPT_CREATE_WORD = "create"
|
|
11
|
+
PULL_REQUEST_SCRIPT_EDIT_WORD = "edit"
|
|
10
12
|
ALL_PYTHON_PROGRAM_NAMES = frozenset({"python", "python3", "py"})
|
|
11
13
|
PROOF_GUIDE_PATH = "~/.claude/docs/rule-guides/proof-before-pull-request.md"
|
|
12
14
|
MISSING_BODY_REASON = (
|
|
@@ -63,18 +65,34 @@ TEST_ONLY_PROOF_REASON = (
|
|
|
63
65
|
" command in backticks and quote the output lines that show the difference. Keep the"
|
|
64
66
|
" test run in the section as extra evidence. Read " + PROOF_GUIDE_PATH + " for the full rule."
|
|
65
67
|
)
|
|
66
|
-
|
|
68
|
+
_LIST_MARKER = r"^\s*(?:(?:[-*>]|\d+[.)])\s*)*"
|
|
69
|
+
NOT_RUN_LINE_PATTERN = _LIST_MARKER + r"not (?:(?:yet|been) )*(?:run|ran|exercised|executed)(?![\w-])"
|
|
70
|
+
UNPROVEN_ADMISSION_PATTERN = (
|
|
71
|
+
r"\b(?:not (?:(?:yet|been|fully) )*(?:proven|tested|verified)|unproven|untested|unverified)\b"
|
|
72
|
+
r"|"
|
|
73
|
+
+ NOT_RUN_LINE_PATTERN
|
|
74
|
+
+ r"|\b(?:has|have)(?: not|n't) (?:yet )?been (?:run|exercised|executed)(?![\w-])"
|
|
75
|
+
)
|
|
76
|
+
MERGE_WORD_PATTERN = r"\bmerg(?:e|ed|es|ing)\b"
|
|
67
77
|
INLINE_CODE_SPAN_PATTERN = r"`[^`\n]*`"
|
|
68
78
|
BLOCKQUOTE_MARKERS = "> "
|
|
69
79
|
ALL_CODE_FENCE_MARKERS = ("```", "~~~")
|
|
70
80
|
UNPROVEN_PROOF_REASON = (
|
|
71
81
|
"This pull request's 'Proof in practice' section says part of the change is not"
|
|
72
82
|
" proven: \"{admission}\". When that part is what the change exists to do, such as"
|
|
73
|
-
" the run on the target that failed,
|
|
74
|
-
"
|
|
83
|
+
" the run on the target that failed, run it first and quote its output in the section."
|
|
84
|
+
" Do not open the pull request, or ask for review of an open one, until that output"
|
|
85
|
+
" is there. When it is a condition the change does not"
|
|
75
86
|
" target, move the line under a 'Not covered' heading at the same level as the proof"
|
|
76
87
|
" heading. Read " + PROOF_GUIDE_PATH + " for the full rule."
|
|
77
88
|
)
|
|
89
|
+
DEFERRED_PROOF_REASON = (
|
|
90
|
+
"This pull request's 'Proof in practice' section mentions a merge: \"{deferral}\". Run that"
|
|
91
|
+
" check on the branch before the review request and quote its output. Put a file or"
|
|
92
|
+
" command name that contains merge in backticks. A step that needs the merged code to"
|
|
93
|
+
" exist goes under a 'Not covered' heading at the same level as the proof heading, as"
|
|
94
|
+
" a line that opens with 'Not run:'. Read " + PROOF_GUIDE_PATH + " for the full rule."
|
|
95
|
+
)
|
|
78
96
|
EXISTING_WORK_HEADING_PATTERN = r"^(#{1,6})[ \t]+existing work[ \t]*#*[ \t]*$"
|
|
79
97
|
EXISTING_WORK_GUIDE_PATH = "~/.claude/docs/rule-guides/explore-thoroughly.md"
|
|
80
98
|
MISSING_EXISTING_WORK_REASON = (
|