claude-dev-env 8.109.6 → 8.109.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/pr-lifecycle/SKILL.md +1 -1
- package/docs/rule-guides/proof-before-pull-request.md +3 -1
- package/hooks/blocking/pull_request_proof.py +258 -50
- package/hooks/blocking/test_pull_request_proof.py +512 -2
- package/hooks/hooks_constants/pull_request_proof_constants.py +19 -1
- package/package.json +1 -1
- package/scripts/durable_post_lint.py +28 -4
- package/scripts/test_durable_post_lint.py +135 -0
|
@@ -22,7 +22,7 @@ User-level rule: applies to **every** git repo that uses GitHub with `gh`. Small
|
|
|
22
22
|
|
|
23
23
|
### Pull request submission rules
|
|
24
24
|
|
|
25
|
-
**Prove the change where it runs before opening a PR.** Read `~/.claude/docs/rule-guides/proof-before-pull-request.md` for the body and hook requirements. A `feat` PR also needs an "Eval" section and a session that invoked `/claude-api build-eval`; read `~/.claude/docs/rule-guides/features-start-with-an-eval.md` before you build the feature. A new workflow that runs only on `workflow_dispatch` cannot run until its file is on the default branch: before the PR, run its `run:` steps with a stub `gh` on PATH, name
|
|
25
|
+
**Prove the change where it runs before opening a PR.** Read `~/.claude/docs/rule-guides/proof-before-pull-request.md` for the body and hook requirements. A `feat` PR also needs an "Eval" section and a session that invoked `/claude-api build-eval`; read `~/.claude/docs/rule-guides/features-start-with-an-eval.md` before you build the feature. A new workflow that runs only on `workflow_dispatch` cannot run until its file is on the default branch: before the PR, run its `run:` steps with a stub `gh` on PATH, name the live run that waits on the merge under an "After merge" heading that names the workflow file in backticks and says who records the run, and dispatch it right after the merge.
|
|
26
26
|
|
|
27
27
|
**Open PRs ready for review; use `--draft` when the owner requests it.** Before you commit copied files or open a PR, list the open PRs that change a file whose path starts with yours, then read each hit, with `gh pr list --state open --limit 1000 --json number,files --jq '.[] | select(any(.files[]; .path | startswith("<path>"))) | .number'`, so you do not duplicate a peer's PR. Each pre-existing failure or remaining defect the PR body names links a tracked issue.
|
|
28
28
|
|
|
@@ -34,11 +34,13 @@ For a change to Claude Code behavior, such as a hook, skill, rule, setting, prom
|
|
|
34
34
|
|
|
35
35
|
Report it in the pull request body under "Proof in practice". List each command you ran, quote the output lines that show the effect, and state the difference between the runs with and without the change. The run on the target the change exists to fix belongs in this section before the pull request opens. List a condition the change does not target, such as an older data shape, under a "Not covered" heading at the same level.
|
|
36
36
|
|
|
37
|
+
A workflow file the pull request adds cannot run until it reaches the default branch. For that run only, add an "After merge" heading to the body. Under it, name the added workflow file in backticks and say who records the first run's link. Prove everything else the workflow does before the pull request opens, such as its script and tests.
|
|
38
|
+
|
|
37
39
|
When you cannot prove it, leave the pull request unopened. Tell the owner in three short sentences what you could not run and which tool would make it possible. A blocked proof is an acceptable result, because it shows where tooling is missing.
|
|
38
40
|
|
|
39
41
|
## Enforcement
|
|
40
42
|
|
|
41
|
-
`hooks/blocking/pull_request_proof.py` reads the body of each new pull request before it opens. `pr_lifecycle_skill_gate.py` runs it on `gh pr create`, the `pull_request.py create` script, and each `mcp__*__create_pull_request` tool. The gate denies the call when the body has no "Proof in practice" heading, or when the text under that heading names no command in backticks. The gate also denies a section whose only commands run a test runner, such as `pytest`, `python -m unittest`, `uv run pytest`, `npm test`, `npx vitest`, `jest`, `go test` or `cargo test`. Quotes around a runner word, interpreter flags before `-m`, and `coverage run -m unittest` are still test runs. A command is an inline backticked span or a `$ ` line inside a fenced block. A span that starts with a path and line, such as `gate.py:12`, is not a command. A span that holds one path or file name, such as `src/x.py`, is not a command unless that span runs a test runner, such as `py.test` or `pytest.exe`. A test run piped into `tee`, `tail`, `head`, `grep` or `cat` is still a test run. The deny quotes the line "Unit tests are welcome as extra evidence. The live run is the proof." and asks for a run of the changed thing the way its user runs it, without the change and with it. The gate also denies a section whose prose says part of the change is not proven, not tested or not verified, with "yet", "been" or "fully" allowed after the "not", or says unproven, untested or unverified. The deny quotes that line, and asks for the run before the pull request opens, or for the line under a "Not covered" heading when the change does not target it. The same denial covers a proof line that opens with "not run", "not exercised" or "not executed" (a "yet" or "been" may follow the "not") or says "has not been run". A proof line with a merge word (merge, merged, merges or merging, so post-merge too) is denied with its own reason: run the check on the branch, and put a file or command name that contains merge in backticks. A step that needs the merged code to exist goes under a "Not covered" heading at the same level as the proof heading, as a line that opens with "not run". `UNPROVEN_ADMISSION_PATTERN` and `MERGE_WORD_PATTERN` list the shapes. Words inside backticks or a fenced block are quoted output and do not count. A section with one command that is not a test run, and no such line, passes. A `gh pr create` with no `--body` or `--body-file` is denied with a request for the body. A body edit through `update_pull_request`, `gh pr edit` or the `pull_request.py edit` script gets the same checks on its proof section that a new pull request gets, so an edit cannot swap in a proof that a new pull request would be denied. An edit with no proof section passes. A body the gate cannot read passes, such as an inline multi-line `--body`, a body from standard input, or a body written through `gh api`. Merges and pushes pass.
|
|
43
|
+
`hooks/blocking/pull_request_proof.py` reads the body of each new pull request before it opens. `pr_lifecycle_skill_gate.py` runs it on `gh pr create`, the `pull_request.py create` script, and each `mcp__*__create_pull_request` tool. The gate denies the call when the body has no "Proof in practice" heading, or when the text under that heading names no command in backticks. The gate also denies a section whose only commands run a test runner, such as `pytest`, `python -m unittest`, `uv run pytest`, `npm test`, `npx vitest`, `jest`, `go test` or `cargo test`. Quotes around a runner word, interpreter flags before `-m`, and `coverage run -m unittest` are still test runs. A command is an inline backticked span or a `$ ` line inside a fenced block. A span that starts with a path and line, such as `gate.py:12`, is not a command. A span that holds one path or file name, such as `src/x.py`, is not a command unless that span runs a test runner, such as `py.test` or `pytest.exe`. A test run piped into `tee`, `tail`, `head`, `grep` or `cat` is still a test run. The deny quotes the line "Unit tests are welcome as extra evidence. The live run is the proof." and asks for a run of the changed thing the way its user runs it, without the change and with it. The gate also denies a section whose prose says part of the change is not proven, not tested or not verified, with "yet", "been" or "fully" allowed after the "not", or says unproven, untested or unverified. The line passes only for a workflow file the branch adds, when the body's "After merge" heading plans that file's first run; `pull_request_proof.py` and `hooks_constants/pull_request_proof_constants.py` hold the exact conditions. Otherwise the deny quotes that line, and asks for the run before the pull request opens, or for the line under a "Not covered" heading when the change does not target it. The same denial covers a proof line that opens with "not run", "not exercised" or "not executed" (a "yet" or "been" may follow the "not") or says "has not been run". A proof line with a merge word (merge, merged, merges or merging, so post-merge too) is denied with its own reason: run the check on the branch, and put a file or command name that contains merge in backticks. A step that needs the merged code to exist goes under a "Not covered" heading at the same level as the proof heading, as a line that opens with "not run". `UNPROVEN_ADMISSION_PATTERN` and `MERGE_WORD_PATTERN` list the shapes. Words inside backticks or a fenced block are quoted output and do not count. A section with one command that is not a test run, and no such line, passes. A `gh pr create` with no `--body` or `--body-file` is denied with a request for the body. A body edit through `update_pull_request`, `gh pr edit` or the `pull_request.py edit` script gets the same checks on its proof section that a new pull request gets, so an edit cannot swap in a proof that a new pull request would be denied. An edit with no proof section passes. A body the gate cannot read passes, such as an inline multi-line `--body`, a body from standard input, or a body written through `gh api`. Merges and pushes pass.
|
|
42
44
|
|
|
43
45
|
When the checked-out branch changes a file people see (`.html`, `.htm`, `.css`, `.scss`, `.tsx`, `.jsx`, `.vue`, `.svelte`), the gate also denies a body whose proof section holds no picture: a markdown image, an image URL, or an Artifact page link that shows the screenshots. Render the change, read the screenshot, and put it in the section, so the reader sees what you looked at. A page the branch only moves to another folder or name that ends the same way, with every byte kept (`R100` in `git diff --name-status`), changes nothing people see, so the gate leaves it out; a moved page that also changed, or a file that becomes a page by a change of name ending, still needs the picture. When git in the session directory cannot list the branch's changed files, the gate asks for the picture as well.
|
|
44
46
|
|
|
@@ -17,6 +17,16 @@ if hooks_directory not in sys.path:
|
|
|
17
17
|
from blocking.followup_pr_dedupe import body_from_arguments, create_tool_fields, title_from_arguments
|
|
18
18
|
from hooks_constants.pr_lifecycle_skill_gate_constants import SHELL_TOOL_NAMES, UPDATE_PULL_REQUEST_TOOL_SUFFIX
|
|
19
19
|
from hooks_constants.pull_request_proof_constants import (
|
|
20
|
+
ADDED_FILE_NAMES_QUERY,
|
|
21
|
+
GIT_URL_SUFFIX,
|
|
22
|
+
REMOTE_LINE_FIELD_COUNT,
|
|
23
|
+
RUN_WORD_PATTERN,
|
|
24
|
+
WORKFLOW_NAME_MATCH_TEMPLATE,
|
|
25
|
+
WORKFLOW_SUBJECT_PATTERN,
|
|
26
|
+
AFTER_MERGE_HEADING_PATTERN,
|
|
27
|
+
AFTER_MERGE_NO_RECORDER_PATTERN,
|
|
28
|
+
AFTER_MERGE_RECORDER_PATTERN,
|
|
29
|
+
AFTER_MERGE_SENTENCE_SEPARATOR_PATTERN,
|
|
20
30
|
ALL_GH_CREATE_WORDS,
|
|
21
31
|
ALL_GH_EDIT_WORDS,
|
|
22
32
|
ALL_PYTHON_PROGRAM_NAMES,
|
|
@@ -50,6 +60,7 @@ from hooks_constants.pull_request_proof_constants import (
|
|
|
50
60
|
MISSING_EXISTING_WORK_REASON,
|
|
51
61
|
MISSING_LOOK_REASON,
|
|
52
62
|
MISSING_PROOF_REASON,
|
|
63
|
+
NEW_WORKFLOW_PATH_PATTERN,
|
|
53
64
|
NEXT_HEADING_PATTERN,
|
|
54
65
|
PROOF_HEADING_PATTERN,
|
|
55
66
|
PURE_RENAME_STATUS,
|
|
@@ -450,17 +461,17 @@ def _prose_lines(section: str) -> Iterator[tuple[str, str]]:
|
|
|
450
461
|
yield each_line, re.sub(INLINE_CODE_SPAN_PATTERN, "", each_line.replace(CODE_FENCE_MARKER, "`"))
|
|
451
462
|
|
|
452
463
|
|
|
453
|
-
def
|
|
454
|
-
"""Return
|
|
464
|
+
def unproven_admissions(section: str) -> list[str]:
|
|
465
|
+
"""Return every prose line of a proof section that says something is not proven.
|
|
455
466
|
|
|
456
467
|
::
|
|
457
468
|
|
|
458
|
-
"Not proven here: a fresh Convert run." -> "Not proven here: a fresh Convert run."
|
|
459
|
-
"The live run is untested." -> "The live run is untested."
|
|
460
|
-
"It has not been tested on the target." -> "It has not been tested on the target."
|
|
461
|
-
"`not proven` printed by the probe" ->
|
|
462
|
-
"> Not proven here: the live run." -> "> Not proven here: the live run."
|
|
463
|
-
"Ran `python run.py`, quoted below." ->
|
|
469
|
+
"Not proven here: a fresh Convert run." -> ["Not proven here: a fresh Convert run."]
|
|
470
|
+
"The live run is untested." -> ["The live run is untested."]
|
|
471
|
+
"It has not been tested on the target." -> ["It has not been tested on the target."]
|
|
472
|
+
"`not proven` printed by the probe" -> []
|
|
473
|
+
"> Not proven here: the live run." -> ["> Not proven here: the live run."]
|
|
474
|
+
"Ran `python run.py`, quoted below." -> []
|
|
464
475
|
|
|
465
476
|
The markers are not proven, not tested and not verified, each with any
|
|
466
477
|
run of yet, been or fully after the not, plus unproven, untested and
|
|
@@ -472,17 +483,15 @@ def unproven_admission(section: str) -> str | None:
|
|
|
472
483
|
Inline code spans and fenced blocks are skipped, so output quoted there
|
|
473
484
|
never counts.
|
|
474
485
|
"""
|
|
475
|
-
return
|
|
476
|
-
|
|
486
|
+
return [
|
|
487
|
+
each_line.strip()
|
|
488
|
+
for each_line, each_prose in _prose_lines(section)
|
|
489
|
+
if re.search(UNPROVEN_ADMISSION_PATTERN, each_prose, re.IGNORECASE)
|
|
490
|
+
]
|
|
477
491
|
|
|
478
|
-
def _first_prose_match(section: str, pattern: str) -> str | None:
|
|
479
|
-
for each_line, each_prose in _prose_lines(section):
|
|
480
|
-
if re.search(pattern, each_prose, re.IGNORECASE):
|
|
481
|
-
return each_line.strip()
|
|
482
|
-
return None
|
|
483
492
|
|
|
484
493
|
|
|
485
|
-
def merge_word_line(section: str) -> str | None:
|
|
494
|
+
def merge_word_line(section: str, all_covered_names: list[str] | None = None) -> str | None:
|
|
486
495
|
"""Return the first prose line of a proof section that holds a merge word, or None.
|
|
487
496
|
|
|
488
497
|
::
|
|
@@ -498,10 +507,18 @@ def merge_word_line(section: str) -> str | None:
|
|
|
498
507
|
so a command or file name that holds the word passes when it sits in
|
|
499
508
|
backticks.
|
|
500
509
|
|
|
510
|
+
A line that is about the first run of a covered new workflow is skipped.
|
|
511
|
+
|
|
501
512
|
Args:
|
|
502
513
|
section: The text under the 'Proof in practice' heading.
|
|
514
|
+
all_covered_names: The added workflow files the After merge section covers.
|
|
503
515
|
"""
|
|
504
|
-
|
|
516
|
+
for each_line, each_prose in _prose_lines(section):
|
|
517
|
+
if re.search(MERGE_WORD_PATTERN, each_prose, re.IGNORECASE) and not _admission_is_about_a_first_run(
|
|
518
|
+
each_line, all_covered_names or []
|
|
519
|
+
):
|
|
520
|
+
return each_line.strip()
|
|
521
|
+
return None
|
|
505
522
|
|
|
506
523
|
|
|
507
524
|
def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
@@ -532,9 +549,14 @@ def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None
|
|
|
532
549
|
flags before -m unittest still count. coverage run -m unittest counts.
|
|
533
550
|
A pipe into tee, tail, head, grep or cat keeps a test run a test run.
|
|
534
551
|
A section whose prose says part of the change is not proven, untested or
|
|
535
|
-
unverified is denied, with the line quoted;
|
|
552
|
+
unverified is denied, with the line quoted; unproven_admissions lists the
|
|
536
553
|
markers. A prose line with a merge word is denied in the proof section;
|
|
537
|
-
merge_word_line lists the words.
|
|
554
|
+
merge_word_line lists the words. The unproven line passes when the pull
|
|
555
|
+
request adds a workflow file the default branch lacks and the body has an
|
|
556
|
+
'After merge' section that names that file in backticks and says who records
|
|
557
|
+
the run, and the line uses the word run outside code spans and says workflow
|
|
558
|
+
or GitHub Actions or names a covered workflow file; see
|
|
559
|
+
_first_unwaived_admission. The same covered line is exempt from the merge word.
|
|
538
560
|
|
|
539
561
|
Args:
|
|
540
562
|
all_payload_fields: The parsed PreToolUse input.
|
|
@@ -545,21 +567,30 @@ def missing_proof_reason(all_payload_fields: Mapping[str, object]) -> str | None
|
|
|
545
567
|
section = proof_section(each_body)
|
|
546
568
|
if not section:
|
|
547
569
|
return MISSING_PROOF_REASON
|
|
548
|
-
section_reason = _proof_section_reason(section)
|
|
570
|
+
section_reason = _proof_section_reason(section, each_body, all_payload_fields)
|
|
549
571
|
if section_reason is not None:
|
|
550
572
|
return section_reason
|
|
551
573
|
return None
|
|
552
574
|
|
|
553
575
|
|
|
554
|
-
def _proof_section_reason(
|
|
576
|
+
def _proof_section_reason(
|
|
577
|
+
section: str, body: str | None = None, all_payload_fields: Mapping[str, object] | None = None
|
|
578
|
+
) -> str | None:
|
|
555
579
|
if COMMAND_MARKER not in section:
|
|
556
580
|
return MISSING_PROOF_REASON
|
|
557
581
|
if _proof_runs_only_tests(section):
|
|
558
582
|
return TEST_ONLY_PROOF_REASON
|
|
559
|
-
|
|
560
|
-
if
|
|
561
|
-
|
|
562
|
-
|
|
583
|
+
all_admissions = unproven_admissions(section)
|
|
584
|
+
if body is None or all_payload_fields is None:
|
|
585
|
+
unwaived_admission = all_admissions[0] if all_admissions else None
|
|
586
|
+
all_covered_names: list[str] = []
|
|
587
|
+
else:
|
|
588
|
+
needs_waiver = bool(all_admissions) or merge_word_line(section) is not None
|
|
589
|
+
all_covered_names = _covered_workflow_names(body, all_payload_fields) if needs_waiver else []
|
|
590
|
+
unwaived_admission = _first_unwaived_admission(all_admissions, all_covered_names)
|
|
591
|
+
if unwaived_admission is not None:
|
|
592
|
+
return UNPROVEN_PROOF_REASON.format(admission=unwaived_admission)
|
|
593
|
+
deferral = merge_word_line(section, all_covered_names)
|
|
563
594
|
return None if deferral is None else DEFERRED_PROOF_REASON.format(deferral=deferral)
|
|
564
595
|
|
|
565
596
|
|
|
@@ -701,15 +732,10 @@ def visible_changed_files(working_directory: str, head_branch: object) -> list[s
|
|
|
701
732
|
working_directory: The session directory the hook payload names.
|
|
702
733
|
head_branch: The head branch a create call names, or None.
|
|
703
734
|
"""
|
|
704
|
-
|
|
735
|
+
merge_base_and_head = _merge_base_and_head(working_directory, head_branch)
|
|
736
|
+
if merge_base_and_head is None:
|
|
705
737
|
return None
|
|
706
|
-
|
|
707
|
-
if head_reference is None:
|
|
708
|
-
return None
|
|
709
|
-
merge_base = _first_merge_base(working_directory, head_reference)
|
|
710
|
-
if merge_base is None:
|
|
711
|
-
return None
|
|
712
|
-
raw_output = _git_output(working_directory, "diff", "--raw", "--no-abbrev", "-M", "-z", merge_base, head_reference)
|
|
738
|
+
raw_output = _git_output(working_directory, "diff", "--raw", "--no-abbrev", "-M", "-z", *merge_base_and_head)
|
|
713
739
|
if raw_output is None:
|
|
714
740
|
return None
|
|
715
741
|
all_changes = _git_changes(raw_output.split("\0"))
|
|
@@ -718,6 +744,16 @@ def visible_changed_files(working_directory: str, head_branch: object) -> list[s
|
|
|
718
744
|
return _visible_paths(all_changes)
|
|
719
745
|
|
|
720
746
|
|
|
747
|
+
def _merge_base_and_head(working_directory: str, head_branch: object) -> tuple[str, str] | None:
|
|
748
|
+
if not working_directory:
|
|
749
|
+
return None
|
|
750
|
+
head_reference = _head_reference(working_directory, head_branch)
|
|
751
|
+
if head_reference is None:
|
|
752
|
+
return None
|
|
753
|
+
merge_base = _first_merge_base(working_directory, head_reference)
|
|
754
|
+
return None if merge_base is None else (merge_base, head_reference)
|
|
755
|
+
|
|
756
|
+
|
|
721
757
|
def _git_changes(all_raw_fields: list[str]) -> list[ChangedFile] | None:
|
|
722
758
|
all_changes = []
|
|
723
759
|
field_index = 0
|
|
@@ -769,6 +805,34 @@ def _visible_paths(all_changes: list[ChangedFile]) -> list[str]:
|
|
|
769
805
|
]
|
|
770
806
|
|
|
771
807
|
|
|
808
|
+
def _workflow_paths(all_added_paths: list[str]) -> list[str]:
|
|
809
|
+
return [each_path for each_path in all_added_paths if re.match(NEW_WORKFLOW_PATH_PATTERN, each_path)]
|
|
810
|
+
|
|
811
|
+
|
|
812
|
+
def added_workflow_files(working_directory: str, head_branch: object) -> list[str] | None:
|
|
813
|
+
"""Return the workflow files the head branch adds, or None when git cannot tell.
|
|
814
|
+
|
|
815
|
+
The head and base are chosen as in visible_changed_files. Only a file git
|
|
816
|
+
reports as added counts, and a rename counts as an add, so a workflow the
|
|
817
|
+
branch edits is left out::
|
|
818
|
+
|
|
819
|
+
branch adds .github/workflows/release.yml -> [".github/workflows/release.yml"]
|
|
820
|
+
branch edits an existing workflow -> []
|
|
821
|
+
named head exists nowhere in the clone -> None
|
|
822
|
+
|
|
823
|
+
Args:
|
|
824
|
+
working_directory: The session directory the hook payload names.
|
|
825
|
+
head_branch: The head branch a create call names, or None.
|
|
826
|
+
"""
|
|
827
|
+
merge_base_and_head = _merge_base_and_head(working_directory, head_branch)
|
|
828
|
+
if merge_base_and_head is None:
|
|
829
|
+
return None
|
|
830
|
+
all_added_paths = _git_lines(
|
|
831
|
+
working_directory, "diff", "--name-only", "--no-renames", "--diff-filter=A", *merge_base_and_head
|
|
832
|
+
)
|
|
833
|
+
return None if all_added_paths is None else _workflow_paths(all_added_paths)
|
|
834
|
+
|
|
835
|
+
|
|
772
836
|
def _gh_api_lines(endpoint: str, query: str) -> list[str] | None:
|
|
773
837
|
try:
|
|
774
838
|
completed = subprocess.run(
|
|
@@ -796,22 +860,20 @@ def _compare_base_branch(repository: str, base_branch: str | None) -> str | None
|
|
|
796
860
|
return all_default_branch_lines[0]
|
|
797
861
|
|
|
798
862
|
|
|
799
|
-
def
|
|
800
|
-
|
|
863
|
+
def _compare_lines(
|
|
864
|
+
repository: str | None, base_branch: str | None, head_branch: str | None, query: str
|
|
865
|
+
) -> list[str] | None:
|
|
866
|
+
if not repository or not head_branch:
|
|
867
|
+
return None
|
|
868
|
+
resolved_base_branch = _compare_base_branch(repository, base_branch)
|
|
869
|
+
if resolved_base_branch is None:
|
|
870
|
+
return None
|
|
871
|
+
return _gh_api_lines(
|
|
801
872
|
COMPARE_ENDPOINT_TEMPLATE.format(
|
|
802
|
-
repository=repository, base_branch=
|
|
873
|
+
repository=repository, base_branch=resolved_base_branch, head_branch=head_branch
|
|
803
874
|
),
|
|
804
|
-
|
|
875
|
+
query,
|
|
805
876
|
)
|
|
806
|
-
if all_status_lines is None:
|
|
807
|
-
return None
|
|
808
|
-
all_changes = _compare_changes(all_status_lines)
|
|
809
|
-
if all_changes is None:
|
|
810
|
-
return None
|
|
811
|
-
all_visible_paths = _visible_paths(all_changes)
|
|
812
|
-
if all_visible_paths or len(all_status_lines) < COMPARE_FILE_LIST_LIMIT:
|
|
813
|
-
return all_visible_paths
|
|
814
|
-
return None
|
|
815
877
|
|
|
816
878
|
|
|
817
879
|
def remote_visible_changed_files(
|
|
@@ -838,12 +900,158 @@ def remote_visible_changed_files(
|
|
|
838
900
|
base_branch: The base branch the create call names, or None.
|
|
839
901
|
head_branch: The head branch the create call names, or None.
|
|
840
902
|
"""
|
|
841
|
-
|
|
903
|
+
all_status_lines = _compare_lines(repository, base_branch, head_branch, COMPARE_FILE_STATUS_QUERY)
|
|
904
|
+
if all_status_lines is None:
|
|
842
905
|
return None
|
|
843
|
-
|
|
844
|
-
if
|
|
906
|
+
all_changes = _compare_changes(all_status_lines)
|
|
907
|
+
if all_changes is None:
|
|
908
|
+
return None
|
|
909
|
+
all_visible_paths = _visible_paths(all_changes)
|
|
910
|
+
if all_visible_paths or len(all_status_lines) < COMPARE_FILE_LIST_LIMIT:
|
|
911
|
+
return all_visible_paths
|
|
912
|
+
return None
|
|
913
|
+
|
|
914
|
+
|
|
915
|
+
def remote_added_workflow_files(
|
|
916
|
+
repository: str | None, base_branch: str | None, head_branch: str | None
|
|
917
|
+
) -> list[str] | None:
|
|
918
|
+
"""Return the workflow files the head branch adds, read from the GitHub compare API, or None.
|
|
919
|
+
|
|
920
|
+
::
|
|
921
|
+
|
|
922
|
+
compare adds .github/workflows/release.yml -> [".github/workflows/release.yml"]
|
|
923
|
+
compare adds only tool.py -> []
|
|
924
|
+
compare returns 300 files and adds no workflow -> None
|
|
925
|
+
repository or head missing -> None
|
|
926
|
+
|
|
927
|
+
A full page of 300 files is the compare API's cap, so a workflow could sit
|
|
928
|
+
on a later page and the answer is None.
|
|
929
|
+
|
|
930
|
+
Args:
|
|
931
|
+
repository: The owner/name the create call names, or None.
|
|
932
|
+
base_branch: The base branch the create call names, or None.
|
|
933
|
+
head_branch: The head branch the create call names, or None.
|
|
934
|
+
"""
|
|
935
|
+
all_lines = _compare_lines(repository, base_branch, head_branch, ADDED_FILE_NAMES_QUERY)
|
|
936
|
+
if not all_lines or not all_lines[0].isdigit():
|
|
845
937
|
return None
|
|
846
|
-
|
|
938
|
+
all_workflow_paths = _workflow_paths(all_lines[1:])
|
|
939
|
+
if all_workflow_paths or int(all_lines[0]) < COMPARE_FILE_LIST_LIMIT:
|
|
940
|
+
return all_workflow_paths
|
|
941
|
+
return None
|
|
942
|
+
|
|
943
|
+
|
|
944
|
+
def _added_workflow_files_for_create(all_payload_fields: Mapping[str, object]) -> list[str]:
|
|
945
|
+
working_directory = str(all_payload_fields.get("cwd") or "")
|
|
946
|
+
head_branch = _named_head_branch(all_payload_fields)
|
|
947
|
+
repository = _named_repository(all_payload_fields)
|
|
948
|
+
all_workflow_paths = None
|
|
949
|
+
if _local_checkout_may_answer(working_directory, repository, head_branch):
|
|
950
|
+
all_workflow_paths = added_workflow_files(working_directory, head_branch)
|
|
951
|
+
if all_workflow_paths is None:
|
|
952
|
+
all_workflow_paths = remote_added_workflow_files(
|
|
953
|
+
repository, _named_base_branch(all_payload_fields), head_branch
|
|
954
|
+
)
|
|
955
|
+
return all_workflow_paths or []
|
|
956
|
+
|
|
957
|
+
|
|
958
|
+
def _local_checkout_may_answer(
|
|
959
|
+
working_directory: str, repository: str | None, head_branch: str | None
|
|
960
|
+
) -> bool:
|
|
961
|
+
return not head_branch or checkout_is_for(working_directory, repository)
|
|
962
|
+
|
|
963
|
+
|
|
964
|
+
def checkout_is_for(working_directory: str, repository: str | None) -> bool:
|
|
965
|
+
if repository is None:
|
|
966
|
+
return True
|
|
967
|
+
all_remote_lines = _git_lines(working_directory, "remote", "-v") if working_directory else None
|
|
968
|
+
all_repository_endings = ("/" + repository.lower(), ":" + repository.lower())
|
|
969
|
+
for each_line in all_remote_lines or []:
|
|
970
|
+
all_fields = each_line.split()
|
|
971
|
+
if len(all_fields) >= REMOTE_LINE_FIELD_COUNT and all_fields[1].lower().rstrip("/").removesuffix(GIT_URL_SUFFIX).endswith(
|
|
972
|
+
all_repository_endings
|
|
973
|
+
):
|
|
974
|
+
return True
|
|
975
|
+
return False
|
|
976
|
+
|
|
977
|
+
|
|
978
|
+
def _first_unwaived_admission(all_admissions: list[str], all_covered_names: list[str]) -> str | None:
|
|
979
|
+
"""Return the first unproven line a new workflow's after-merge run does not explain, or None.
|
|
980
|
+
|
|
981
|
+
::
|
|
982
|
+
|
|
983
|
+
"Not proven here: the live GitHub Actions run." with a covered new workflow -> None
|
|
984
|
+
"Untested on Windows." with a covered new workflow -> that line
|
|
985
|
+
"Not proven here: the live GitHub Actions run." with no covered workflow -> that line
|
|
986
|
+
no unproven line -> None
|
|
987
|
+
|
|
988
|
+
Each admission line must use the word run and also say GitHub Actions or
|
|
989
|
+
workflow, or name a covered workflow file, so an admission about anything
|
|
990
|
+
else keeps the refusal. The line returned is the one that failed.
|
|
991
|
+
|
|
992
|
+
Args:
|
|
993
|
+
all_admissions: The proof section's unproven lines.
|
|
994
|
+
all_covered_names: The added workflow files the After merge section covers.
|
|
995
|
+
"""
|
|
996
|
+
return next(
|
|
997
|
+
(each for each in all_admissions if not _admission_is_about_a_first_run(each, all_covered_names)), None
|
|
998
|
+
)
|
|
999
|
+
|
|
1000
|
+
|
|
1001
|
+
def _admission_is_about_a_first_run(admission: str, all_workflow_names: list[str]) -> bool:
|
|
1002
|
+
admission_prose = re.sub(INLINE_CODE_SPAN_PATTERN, "", admission)
|
|
1003
|
+
if not all_workflow_names or re.search(RUN_WORD_PATTERN, admission_prose, re.IGNORECASE) is None:
|
|
1004
|
+
return False
|
|
1005
|
+
return re.search(WORKFLOW_SUBJECT_PATTERN, admission_prose, re.IGNORECASE) is not None or any(
|
|
1006
|
+
re.search(WORKFLOW_NAME_MATCH_TEMPLATE.format(workflow_name=re.escape(each_name)), admission, re.IGNORECASE)
|
|
1007
|
+
for each_name in all_workflow_names
|
|
1008
|
+
)
|
|
1009
|
+
|
|
1010
|
+
|
|
1011
|
+
def _covered_workflow_names(body: str, all_payload_fields: Mapping[str, object]) -> list[str]:
|
|
1012
|
+
"""Return the added workflow files whose first run the body plans for after the merge.
|
|
1013
|
+
|
|
1014
|
+
::
|
|
1015
|
+
|
|
1016
|
+
adds release.yml, "## After merge" names `release.yml` and "records the run link" -> ["release.yml"]
|
|
1017
|
+
the branch adds no workflow file, or git and the compare API cannot tell -> []
|
|
1018
|
+
|
|
1019
|
+
A workflow added by a pull request runs only from the default branch, so
|
|
1020
|
+
its live run cannot come before the merge. The section must name the added
|
|
1021
|
+
workflow file in backticks, by file name or by its full path, and say who
|
|
1022
|
+
records the run or its link in prose. Code spans and fenced blocks do not
|
|
1023
|
+
count as that prose. A sentence with a negation ahead of the verb does not count.
|
|
1024
|
+
|
|
1025
|
+
Args:
|
|
1026
|
+
body: The pull request body.
|
|
1027
|
+
all_payload_fields: The parsed PreToolUse input.
|
|
1028
|
+
"""
|
|
1029
|
+
after_merge = _section_text(body, AFTER_MERGE_HEADING_PATTERN)
|
|
1030
|
+
if not after_merge:
|
|
1031
|
+
return []
|
|
1032
|
+
outside_fence_text = _text_outside_fences(after_merge)
|
|
1033
|
+
if not _prose_names_a_recorder(re.sub(INLINE_CODE_SPAN_PATTERN, "", outside_fence_text)):
|
|
1034
|
+
return []
|
|
1035
|
+
all_names = []
|
|
1036
|
+
for each_path in _added_workflow_files_for_create(all_payload_fields):
|
|
1037
|
+
each_name = each_path.rsplit("/", 1)[-1]
|
|
1038
|
+
if f"`{each_path}`" in outside_fence_text or f"`{each_name}`" in outside_fence_text:
|
|
1039
|
+
all_names.append(each_name)
|
|
1040
|
+
return all_names
|
|
1041
|
+
|
|
1042
|
+
|
|
1043
|
+
def _text_outside_fences(section: str) -> str:
|
|
1044
|
+
return LINE_SEPARATOR.join(
|
|
1045
|
+
each_line for each_line, each_is_inside_fence in _lines_with_fence_state(section) if not each_is_inside_fence
|
|
1046
|
+
)
|
|
1047
|
+
|
|
1048
|
+
|
|
1049
|
+
def _prose_names_a_recorder(prose: str) -> bool:
|
|
1050
|
+
return any(
|
|
1051
|
+
re.search(AFTER_MERGE_RECORDER_PATTERN, each_sentence, re.IGNORECASE) is not None
|
|
1052
|
+
and re.search(AFTER_MERGE_NO_RECORDER_PATTERN, each_sentence, re.IGNORECASE) is None
|
|
1053
|
+
for each_sentence in re.split(AFTER_MERGE_SENTENCE_SEPARATOR_PATTERN, prose)
|
|
1054
|
+
)
|
|
847
1055
|
|
|
848
1056
|
|
|
849
1057
|
def missing_look_reason(all_payload_fields: Mapping[str, object]) -> str | None:
|
|
@@ -377,6 +377,19 @@ def test_should_deny_a_page_change_whose_proof_shows_no_picture(tmp_path: Path)
|
|
|
377
377
|
assert "page.html" in reason
|
|
378
378
|
|
|
379
379
|
|
|
380
|
+
def test_should_read_the_checkout_when_the_call_names_a_repository_but_no_head(
|
|
381
|
+
tmp_path: Path,
|
|
382
|
+
) -> None:
|
|
383
|
+
clone_path = _repository_with_change(tmp_path, "tool.py")
|
|
384
|
+
body_path = tmp_path / "body.md"
|
|
385
|
+
body_path.write_text(PROVEN_BODY, encoding="utf-8")
|
|
386
|
+
payload = _shell(
|
|
387
|
+
f"gh pr create --title t --body-file {body_path} --repo o/r --base main",
|
|
388
|
+
clone_path,
|
|
389
|
+
)
|
|
390
|
+
assert gate.missing_look_reason(payload) is None
|
|
391
|
+
|
|
392
|
+
|
|
380
393
|
def test_should_allow_a_page_change_whose_proof_links_a_screenshot(tmp_path: Path) -> None:
|
|
381
394
|
clone_path = _repository_with_change(tmp_path, "page.html")
|
|
382
395
|
body = PROVEN_BODY + "\n"
|
|
@@ -411,9 +424,11 @@ def _fake_gh_api(all_responses: dict[str, list[str] | None], all_endpoints: list
|
|
|
411
424
|
return fake_gh_api_lines
|
|
412
425
|
|
|
413
426
|
|
|
414
|
-
def _shell_create_elsewhere(
|
|
427
|
+
def _shell_create_elsewhere(
|
|
428
|
+
tmp_path: Path, extra_flags: str, body: str = PROVEN_BODY
|
|
429
|
+
) -> dict[str, object]:
|
|
415
430
|
body_path = tmp_path / "body.md"
|
|
416
|
-
body_path.write_text(
|
|
431
|
+
body_path.write_text(body, encoding="utf-8")
|
|
417
432
|
return _shell(f"gh pr create --title t --body-file {body_path} {extra_flags}", tmp_path)
|
|
418
433
|
|
|
419
434
|
|
|
@@ -840,6 +855,501 @@ def test_is_create_call_should_name_shell_and_mcp_creates() -> None:
|
|
|
840
855
|
assert not gate.is_create_call({"tool_name": "Read", "tool_input": {"file_path": "a.py"}})
|
|
841
856
|
|
|
842
857
|
|
|
858
|
+
WORKFLOW_DIRECTORY = ".github/workflows"
|
|
859
|
+
NEW_WORKFLOW_NAME = "release-hold.yml"
|
|
860
|
+
UNPROVEN_RUN_LINE = "Not proven here: the live GitHub Actions run."
|
|
861
|
+
AFTER_MERGE_SECTION = (
|
|
862
|
+
"\n## After merge\n\n- The first owner approval runs `release-hold.yml`."
|
|
863
|
+
" The agent driving this change records that run link on this pull request.\n"
|
|
864
|
+
)
|
|
865
|
+
|
|
866
|
+
|
|
867
|
+
def _workflow_body(after_merge_section: str) -> str:
|
|
868
|
+
return (
|
|
869
|
+
"## Proof in practice\nRan `python run.py`.\n"
|
|
870
|
+
+ UNPROVEN_RUN_LINE
|
|
871
|
+
+ "\n"
|
|
872
|
+
+ after_merge_section
|
|
873
|
+
)
|
|
874
|
+
|
|
875
|
+
|
|
876
|
+
def _repository_with_workflow_change(
|
|
877
|
+
tmp_path: Path, changed_workflow_name: str
|
|
878
|
+
) -> Path:
|
|
879
|
+
origin_path = tmp_path / "origin"
|
|
880
|
+
clone_path = tmp_path / "clone"
|
|
881
|
+
git_identity = ["-c", "user.name=t", "-c", "user.email=t@example.com"]
|
|
882
|
+
subprocess.run(["git", "init", "-q", "-b", "main", str(origin_path)], check=True)
|
|
883
|
+
(origin_path / WORKFLOW_DIRECTORY).mkdir(parents=True)
|
|
884
|
+
(origin_path / WORKFLOW_DIRECTORY / "ci.yml").write_text(
|
|
885
|
+
"name: ci\n", encoding="utf-8"
|
|
886
|
+
)
|
|
887
|
+
subprocess.run(["git", "-C", str(origin_path), "add", "."], check=True)
|
|
888
|
+
subprocess.run(
|
|
889
|
+
["git", "-C", str(origin_path), *git_identity, "commit", "-q", "-m", "base"],
|
|
890
|
+
check=True,
|
|
891
|
+
)
|
|
892
|
+
subprocess.run(
|
|
893
|
+
["git", "clone", "-q", str(origin_path), str(clone_path)], check=True
|
|
894
|
+
)
|
|
895
|
+
subprocess.run(
|
|
896
|
+
["git", "-C", str(clone_path), "checkout", "-q", "-b", "feature"], check=True
|
|
897
|
+
)
|
|
898
|
+
(clone_path / WORKFLOW_DIRECTORY / changed_workflow_name).write_text(
|
|
899
|
+
"name: changed\n", encoding="utf-8"
|
|
900
|
+
)
|
|
901
|
+
subprocess.run(["git", "-C", str(clone_path), "add", "."], check=True)
|
|
902
|
+
subprocess.run(
|
|
903
|
+
["git", "-C", str(clone_path), *git_identity, "commit", "-q", "-m", "change"],
|
|
904
|
+
check=True,
|
|
905
|
+
)
|
|
906
|
+
return clone_path
|
|
907
|
+
|
|
908
|
+
|
|
909
|
+
def test_unproven_line_should_pass_when_a_new_workflow_names_its_after_merge_run(
|
|
910
|
+
tmp_path: Path,
|
|
911
|
+
) -> None:
|
|
912
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
913
|
+
payload = _mcp_create_on_branch(_workflow_body(AFTER_MERGE_SECTION), clone_path)
|
|
914
|
+
assert gate.missing_proof_reason(payload) is None
|
|
915
|
+
|
|
916
|
+
|
|
917
|
+
def test_unproven_line_should_stay_denied_when_the_branch_only_edits_an_existing_workflow(
|
|
918
|
+
tmp_path: Path,
|
|
919
|
+
) -> None:
|
|
920
|
+
clone_path = _repository_with_workflow_change(tmp_path, "ci.yml")
|
|
921
|
+
after_merge_section = AFTER_MERGE_SECTION.replace(NEW_WORKFLOW_NAME, "ci.yml")
|
|
922
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
923
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
924
|
+
admission=UNPROVEN_RUN_LINE
|
|
925
|
+
)
|
|
926
|
+
|
|
927
|
+
|
|
928
|
+
@pytest.mark.parametrize(
|
|
929
|
+
"after_merge_section",
|
|
930
|
+
[
|
|
931
|
+
"",
|
|
932
|
+
"\n## After merge\n\n- The first owner approval runs `other.yml`. The agent records the link.\n",
|
|
933
|
+
"\n## After merge\n\n- The first owner approval runs `release-hold.yml`.\n",
|
|
934
|
+
"\n## After merge\n\n- The first owner approval runs release-hold.yml. The agent records the link.\n",
|
|
935
|
+
],
|
|
936
|
+
ids=[
|
|
937
|
+
"no-section",
|
|
938
|
+
"other-workflow-named",
|
|
939
|
+
"no-recorder",
|
|
940
|
+
"workflow-not-in-backticks",
|
|
941
|
+
],
|
|
942
|
+
)
|
|
943
|
+
def test_unproven_line_should_stay_denied_when_the_after_merge_section_falls_short(
|
|
944
|
+
tmp_path: Path, after_merge_section: str
|
|
945
|
+
) -> None:
|
|
946
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
947
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
948
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
949
|
+
admission=UNPROVEN_RUN_LINE
|
|
950
|
+
)
|
|
951
|
+
|
|
952
|
+
|
|
953
|
+
def test_unproven_line_should_stay_denied_when_git_cannot_read_the_branch(
|
|
954
|
+
tmp_path: Path,
|
|
955
|
+
) -> None:
|
|
956
|
+
payload = _mcp_create_on_branch(_workflow_body(AFTER_MERGE_SECTION), tmp_path)
|
|
957
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
958
|
+
admission=UNPROVEN_RUN_LINE
|
|
959
|
+
)
|
|
960
|
+
|
|
961
|
+
|
|
962
|
+
def test_added_workflow_files_should_list_only_files_the_branch_adds(
|
|
963
|
+
tmp_path: Path,
|
|
964
|
+
) -> None:
|
|
965
|
+
added_clone = _repository_with_workflow_change(
|
|
966
|
+
tmp_path / "added", NEW_WORKFLOW_NAME
|
|
967
|
+
)
|
|
968
|
+
edited_clone = _repository_with_workflow_change(tmp_path / "edited", "ci.yml")
|
|
969
|
+
assert gate.added_workflow_files(str(added_clone), "feature") == [
|
|
970
|
+
f"{WORKFLOW_DIRECTORY}/{NEW_WORKFLOW_NAME}"
|
|
971
|
+
]
|
|
972
|
+
assert gate.added_workflow_files(str(edited_clone), "feature") == []
|
|
973
|
+
assert gate.added_workflow_files(str(added_clone), "other-branch") is None
|
|
974
|
+
|
|
975
|
+
|
|
976
|
+
def _workflow_compare_lines(count: int, all_added_paths: list[str]) -> list[str]:
|
|
977
|
+
return [str(count), *all_added_paths]
|
|
978
|
+
|
|
979
|
+
|
|
980
|
+
def test_unproven_line_should_pass_on_the_compare_api_when_a_new_workflow_is_added(
|
|
981
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
982
|
+
) -> None:
|
|
983
|
+
all_endpoints: list[str] = []
|
|
984
|
+
compare_endpoint = "repos/o/r/compare/main...feature"
|
|
985
|
+
added_path = f"{WORKFLOW_DIRECTORY}/{NEW_WORKFLOW_NAME}"
|
|
986
|
+
all_responses = {compare_endpoint: _workflow_compare_lines(2, [added_path])}
|
|
987
|
+
monkeypatch.setattr(
|
|
988
|
+
gate, "_gh_api_lines", _fake_gh_api(all_responses, all_endpoints)
|
|
989
|
+
)
|
|
990
|
+
payload = _shell_create_elsewhere(
|
|
991
|
+
tmp_path,
|
|
992
|
+
"--repo o/r --base main --head feature",
|
|
993
|
+
_workflow_body(AFTER_MERGE_SECTION),
|
|
994
|
+
)
|
|
995
|
+
assert gate.remote_added_workflow_files("o/r", "main", "feature") == [added_path]
|
|
996
|
+
assert gate.missing_proof_reason(payload) is None
|
|
997
|
+
|
|
998
|
+
|
|
999
|
+
def test_remote_added_workflow_files_should_report_none_when_a_full_page_lists_no_workflow(
|
|
1000
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
1001
|
+
) -> None:
|
|
1002
|
+
compare_endpoint = "repos/o/r/compare/main...feature"
|
|
1003
|
+
all_responses = {
|
|
1004
|
+
compare_endpoint: _workflow_compare_lines(
|
|
1005
|
+
COMPARE_FILE_LIST_LIMIT, ["pkg/module_0.py"]
|
|
1006
|
+
),
|
|
1007
|
+
}
|
|
1008
|
+
monkeypatch.setattr(gate, "_gh_api_lines", _fake_gh_api(all_responses, []))
|
|
1009
|
+
assert gate.remote_added_workflow_files("o/r", "main", "feature") is None
|
|
1010
|
+
all_responses[compare_endpoint] = _workflow_compare_lines(
|
|
1011
|
+
COMPARE_FILE_LIST_LIMIT - 1, ["pkg/module_0.py"]
|
|
1012
|
+
)
|
|
1013
|
+
assert gate.remote_added_workflow_files("o/r", "main", "feature") == []
|
|
1014
|
+
assert gate.remote_added_workflow_files(None, "main", "feature") is None
|
|
1015
|
+
|
|
1016
|
+
|
|
1017
|
+
def test_unproven_line_should_stay_denied_when_it_is_about_something_other_than_a_run(tmp_path: Path) -> None:
|
|
1018
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1019
|
+
unrelated_line = "Untested on Windows."
|
|
1020
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, unrelated_line)
|
|
1021
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1022
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=unrelated_line)
|
|
1023
|
+
|
|
1024
|
+
|
|
1025
|
+
def test_unproven_line_should_stay_denied_and_quote_the_unrelated_line_when_one_of_two_lines_is_unrelated(tmp_path: Path) -> None:
|
|
1026
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1027
|
+
unrelated_line = "Untested on Windows."
|
|
1028
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(
|
|
1029
|
+
UNPROVEN_RUN_LINE, UNPROVEN_RUN_LINE + "\n" + unrelated_line
|
|
1030
|
+
)
|
|
1031
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1032
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=unrelated_line)
|
|
1033
|
+
|
|
1034
|
+
|
|
1035
|
+
@pytest.mark.parametrize(
|
|
1036
|
+
"recorder_sentence",
|
|
1037
|
+
["Nobody records that run link.", "No one will record the run link.", "The agent does not record the run link."],
|
|
1038
|
+
)
|
|
1039
|
+
def test_unproven_line_should_stay_denied_when_the_after_merge_section_denies_a_recorder(
|
|
1040
|
+
tmp_path: Path, recorder_sentence: str
|
|
1041
|
+
) -> None:
|
|
1042
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1043
|
+
after_merge_section = f"\n## After merge\n\n- The first owner approval runs `release-hold.yml`. {recorder_sentence}\n"
|
|
1044
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1045
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=UNPROVEN_RUN_LINE)
|
|
1046
|
+
|
|
1047
|
+
|
|
1048
|
+
def test_unproven_admissions_should_list_every_prose_line_that_says_something_is_unproven() -> None:
|
|
1049
|
+
section = "Ran `python run.py`.\nNot proven here: the live run.\nUntested on Windows.\n"
|
|
1050
|
+
assert gate.unproven_admissions(section) == ["Not proven here: the live run.", "Untested on Windows."]
|
|
1051
|
+
assert gate.unproven_admissions(section)[0] == "Not proven here: the live run."
|
|
1052
|
+
assert gate.unproven_admissions("Ran `python run.py`.\n") == []
|
|
1053
|
+
|
|
1054
|
+
|
|
1055
|
+
def test_unproven_line_should_pass_when_the_after_merge_section_names_the_full_workflow_path(
|
|
1056
|
+
tmp_path: Path,
|
|
1057
|
+
) -> None:
|
|
1058
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1059
|
+
after_merge_section = AFTER_MERGE_SECTION.replace(
|
|
1060
|
+
NEW_WORKFLOW_NAME, f"{WORKFLOW_DIRECTORY}/{NEW_WORKFLOW_NAME}"
|
|
1061
|
+
)
|
|
1062
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1063
|
+
assert gate.missing_proof_reason(payload) is None
|
|
1064
|
+
|
|
1065
|
+
|
|
1066
|
+
def test_added_workflow_files_should_count_a_renamed_workflow_as_added(
|
|
1067
|
+
tmp_path: Path,
|
|
1068
|
+
) -> None:
|
|
1069
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1070
|
+
subprocess.run(
|
|
1071
|
+
[
|
|
1072
|
+
"git",
|
|
1073
|
+
"-C",
|
|
1074
|
+
str(clone_path),
|
|
1075
|
+
"mv",
|
|
1076
|
+
f"{WORKFLOW_DIRECTORY}/ci.yml",
|
|
1077
|
+
f"{WORKFLOW_DIRECTORY}/renamed.yml",
|
|
1078
|
+
],
|
|
1079
|
+
check=True,
|
|
1080
|
+
)
|
|
1081
|
+
git_identity = ["-c", "user.name=t", "-c", "user.email=t@example.com"]
|
|
1082
|
+
subprocess.run(
|
|
1083
|
+
["git", "-C", str(clone_path), *git_identity, "commit", "-q", "-m", "rename"],
|
|
1084
|
+
check=True,
|
|
1085
|
+
)
|
|
1086
|
+
assert f"{WORKFLOW_DIRECTORY}/renamed.yml" in gate.added_workflow_files(
|
|
1087
|
+
str(clone_path), "feature"
|
|
1088
|
+
)
|
|
1089
|
+
|
|
1090
|
+
|
|
1091
|
+
@pytest.mark.parametrize(
|
|
1092
|
+
"unrelated_line",
|
|
1093
|
+
[
|
|
1094
|
+
"Not proven here: the live run of the Convert change on Windows.",
|
|
1095
|
+
"Not proven here: release-hold.yml on macOS.",
|
|
1096
|
+
"Untested on Windows (`python run.py`) for the workflow helper.",
|
|
1097
|
+
],
|
|
1098
|
+
ids=["run-without-a-workflow", "workflow-without-a-run", "run-only-in-a-code-span"],
|
|
1099
|
+
)
|
|
1100
|
+
def test_unproven_line_should_stay_denied_when_it_is_not_about_a_workflow_run(
|
|
1101
|
+
tmp_path: Path, unrelated_line: str
|
|
1102
|
+
) -> None:
|
|
1103
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1104
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, unrelated_line)
|
|
1105
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1106
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
1107
|
+
admission=unrelated_line
|
|
1108
|
+
)
|
|
1109
|
+
|
|
1110
|
+
|
|
1111
|
+
def test_merge_word_line_should_pass_when_it_is_about_the_first_run_of_a_covered_workflow(
|
|
1112
|
+
tmp_path: Path,
|
|
1113
|
+
) -> None:
|
|
1114
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1115
|
+
merge_line = "The live GitHub Actions run waits until this merges."
|
|
1116
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, merge_line)
|
|
1117
|
+
assert gate.missing_proof_reason(_mcp_create_on_branch(body, clone_path)) is None
|
|
1118
|
+
assert gate.missing_proof_reason(_mcp_create(body)) == DEFERRED_PROOF_REASON.format(deferral=merge_line)
|
|
1119
|
+
|
|
1120
|
+
|
|
1121
|
+
def test_merge_word_line_should_stay_denied_when_it_is_not_about_a_workflow_run(
|
|
1122
|
+
tmp_path: Path,
|
|
1123
|
+
) -> None:
|
|
1124
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1125
|
+
merge_line = "The Convert change is checked after the merge."
|
|
1126
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, merge_line)
|
|
1127
|
+
assert gate.missing_proof_reason(_mcp_create_on_branch(body, clone_path)) == DEFERRED_PROOF_REASON.format(
|
|
1128
|
+
deferral=merge_line
|
|
1129
|
+
)
|
|
1130
|
+
|
|
1131
|
+
|
|
1132
|
+
def test_unproven_line_should_stay_denied_when_a_run_line_names_no_workflow(
|
|
1133
|
+
tmp_path: Path,
|
|
1134
|
+
) -> None:
|
|
1135
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1136
|
+
unrelated_line = "Untested: the Convert run on Windows."
|
|
1137
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(
|
|
1138
|
+
UNPROVEN_RUN_LINE, unrelated_line
|
|
1139
|
+
)
|
|
1140
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1141
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
1142
|
+
admission=unrelated_line
|
|
1143
|
+
)
|
|
1144
|
+
|
|
1145
|
+
|
|
1146
|
+
def test_unproven_line_should_pass_when_a_negation_sits_in_an_earlier_clause(
|
|
1147
|
+
tmp_path: Path,
|
|
1148
|
+
) -> None:
|
|
1149
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1150
|
+
after_merge_section = "\n## After merge\n\n- `release-hold.yml` does not run on pull requests; the agent records the run link.\n"
|
|
1151
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1152
|
+
assert gate.missing_proof_reason(payload) is None
|
|
1153
|
+
|
|
1154
|
+
|
|
1155
|
+
@pytest.mark.parametrize(
|
|
1156
|
+
"after_merge_section",
|
|
1157
|
+
[
|
|
1158
|
+
"\n## After merge\n\n- Runs `release-hold.yml`. There are no records of it yet.\n",
|
|
1159
|
+
"\n## After merge\n\n- The first owner approval runs `pre-release-hold.yml`. The agent records the run link.\n",
|
|
1160
|
+
"\n## After merge\n\n- The first owner approval runs `release-hold.yml`.\n\n```\nagent records the run link\n```\n",
|
|
1161
|
+
"\n## After merge\n\n- The first owner approval runs `release-hold.yml`. The `agent records the run link`.\n",
|
|
1162
|
+
],
|
|
1163
|
+
ids=[
|
|
1164
|
+
"noun-records",
|
|
1165
|
+
"name-is-a-longer-file",
|
|
1166
|
+
"recorder-only-in-fence",
|
|
1167
|
+
"recorder-only-in-code-span",
|
|
1168
|
+
],
|
|
1169
|
+
)
|
|
1170
|
+
def test_unproven_line_should_stay_denied_when_the_after_merge_evidence_is_not_prose_about_this_workflow(
|
|
1171
|
+
tmp_path: Path, after_merge_section: str
|
|
1172
|
+
) -> None:
|
|
1173
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1174
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1175
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(
|
|
1176
|
+
admission=UNPROVEN_RUN_LINE
|
|
1177
|
+
)
|
|
1178
|
+
|
|
1179
|
+
|
|
1180
|
+
def test_unproven_line_should_use_the_compare_api_when_a_local_checkout_lists_no_added_workflow(
|
|
1181
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
1182
|
+
) -> None:
|
|
1183
|
+
compare_endpoint = "repos/o/r/compare/main...feature"
|
|
1184
|
+
added_path = f"{WORKFLOW_DIRECTORY}/{NEW_WORKFLOW_NAME}"
|
|
1185
|
+
all_responses = {compare_endpoint: _workflow_compare_lines(2, [added_path])}
|
|
1186
|
+
monkeypatch.setattr(gate, "_gh_api_lines", _fake_gh_api(all_responses, []))
|
|
1187
|
+
unrelated_checkout = _repository_with_change(tmp_path, "tool.py")
|
|
1188
|
+
payload = {
|
|
1189
|
+
"tool_name": "mcp__github__create_pull_request",
|
|
1190
|
+
"tool_input": {
|
|
1191
|
+
"owner": "o",
|
|
1192
|
+
"repo": "r",
|
|
1193
|
+
"base": "main",
|
|
1194
|
+
"head": "feature",
|
|
1195
|
+
"title": "t",
|
|
1196
|
+
"body": _workflow_body(AFTER_MERGE_SECTION),
|
|
1197
|
+
},
|
|
1198
|
+
"cwd": str(unrelated_checkout),
|
|
1199
|
+
}
|
|
1200
|
+
assert gate.missing_proof_reason(payload) is None
|
|
1201
|
+
|
|
1202
|
+
|
|
1203
|
+
@pytest.mark.parametrize(
|
|
1204
|
+
"after_merge_section",
|
|
1205
|
+
[
|
|
1206
|
+
"\n## After merge\n\n- The first owner approval runs `other/release-hold.yml`. The agent records the run link.\n",
|
|
1207
|
+
"\n## After merge\n\n- The first owner approval runs `release-hold.yml`. Nobody has recorded the run link.\n",
|
|
1208
|
+
],
|
|
1209
|
+
ids=["same-name-in-another-directory", "past-tense-negated-recorder"],
|
|
1210
|
+
)
|
|
1211
|
+
def test_unproven_line_should_stay_denied_for_a_wrong_path_or_a_negated_past_tense_recorder(
|
|
1212
|
+
tmp_path: Path, after_merge_section: str
|
|
1213
|
+
) -> None:
|
|
1214
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1215
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1216
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=UNPROVEN_RUN_LINE)
|
|
1217
|
+
|
|
1218
|
+
|
|
1219
|
+
def test_unproven_line_should_stay_denied_when_it_is_about_another_workflow_the_branch_changes(
|
|
1220
|
+
tmp_path: Path,
|
|
1221
|
+
) -> None:
|
|
1222
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1223
|
+
unrelated_line = "The ci.yml workflow change is untested on Windows runners."
|
|
1224
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, unrelated_line)
|
|
1225
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1226
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=unrelated_line)
|
|
1227
|
+
|
|
1228
|
+
|
|
1229
|
+
def test_unproven_line_should_ignore_a_checkout_that_belongs_to_another_repository(
|
|
1230
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
1231
|
+
) -> None:
|
|
1232
|
+
other_checkout = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1233
|
+
subprocess.run(
|
|
1234
|
+
["git", "-C", str(other_checkout), "remote", "set-url", "origin", "https://github.com/x/other.git"],
|
|
1235
|
+
check=True,
|
|
1236
|
+
)
|
|
1237
|
+
compare_endpoint = "repos/o/r/compare/main...feature"
|
|
1238
|
+
monkeypatch.setattr(gate, "_gh_api_lines", _fake_gh_api({compare_endpoint: ["1", "tool.py"]}, []))
|
|
1239
|
+
payload = {
|
|
1240
|
+
"tool_name": "mcp__github__create_pull_request",
|
|
1241
|
+
"tool_input": {
|
|
1242
|
+
"owner": "o",
|
|
1243
|
+
"repo": "r",
|
|
1244
|
+
"base": "main",
|
|
1245
|
+
"head": "feature",
|
|
1246
|
+
"title": "t",
|
|
1247
|
+
"body": _workflow_body(AFTER_MERGE_SECTION),
|
|
1248
|
+
},
|
|
1249
|
+
"cwd": str(other_checkout),
|
|
1250
|
+
}
|
|
1251
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=UNPROVEN_RUN_LINE)
|
|
1252
|
+
|
|
1253
|
+
|
|
1254
|
+
def test_unproven_line_should_pass_when_a_negation_sits_in_an_earlier_comma_clause(tmp_path: Path) -> None:
|
|
1255
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1256
|
+
after_merge_section = (
|
|
1257
|
+
"\n## After merge\n\n- `release-hold.yml` cannot run on pull requests, so the agent records the run link.\n"
|
|
1258
|
+
)
|
|
1259
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1260
|
+
assert gate.missing_proof_reason(payload) is None
|
|
1261
|
+
|
|
1262
|
+
|
|
1263
|
+
def test_unproven_line_should_pass_when_one_sentence_names_a_recorder_beside_a_negated_one(
|
|
1264
|
+
tmp_path: Path,
|
|
1265
|
+
) -> None:
|
|
1266
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1267
|
+
after_merge_section = (
|
|
1268
|
+
"\n## After merge\n\n- Nobody records the run link for ci.yml. The agent records the run link of `release-hold.yml`.\n"
|
|
1269
|
+
)
|
|
1270
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1271
|
+
assert gate.missing_proof_reason(payload) is None
|
|
1272
|
+
|
|
1273
|
+
|
|
1274
|
+
def test_unproven_line_should_stay_denied_when_the_recorder_records_nothing(tmp_path: Path) -> None:
|
|
1275
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1276
|
+
after_merge_section = "\n## After merge\n\n- Runs `release-hold.yml`. The agent records nothing; the run link is lost.\n"
|
|
1277
|
+
payload = _mcp_create_on_branch(_workflow_body(after_merge_section), clone_path)
|
|
1278
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=UNPROVEN_RUN_LINE)
|
|
1279
|
+
|
|
1280
|
+
|
|
1281
|
+
def test_unproven_line_should_stay_denied_when_the_line_is_about_another_live_run(tmp_path: Path) -> None:
|
|
1282
|
+
clone_path = _repository_with_workflow_change(tmp_path, NEW_WORKFLOW_NAME)
|
|
1283
|
+
unrelated_line = "Not proven here: the live Convert run on Windows."
|
|
1284
|
+
body = _workflow_body(AFTER_MERGE_SECTION).replace(UNPROVEN_RUN_LINE, unrelated_line)
|
|
1285
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1286
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=unrelated_line)
|
|
1287
|
+
|
|
1288
|
+
|
|
1289
|
+
@pytest.mark.parametrize(
|
|
1290
|
+
("origin_url", "repository", "expected"),
|
|
1291
|
+
[
|
|
1292
|
+
("https://github.com/o/r.git", "o/r", True),
|
|
1293
|
+
("git@github.com:o/r.git", "o/r", True),
|
|
1294
|
+
("https://github.com/o/r/", "o/r", True),
|
|
1295
|
+
("https://github.com/foo/r", "o/r", False),
|
|
1296
|
+
("https://github.com/o/r", None, True),
|
|
1297
|
+
],
|
|
1298
|
+
)
|
|
1299
|
+
def test_checkout_should_belong_to_the_target_repository_only_on_a_full_name_match(
|
|
1300
|
+
tmp_path: Path, origin_url: str, repository: str | None, expected: bool
|
|
1301
|
+
) -> None:
|
|
1302
|
+
subprocess.run(["git", "init", "-q", "-b", "main", str(tmp_path)], check=True)
|
|
1303
|
+
subprocess.run(["git", "-C", str(tmp_path), "remote", "add", "origin", origin_url], check=True)
|
|
1304
|
+
assert gate.checkout_is_for(str(tmp_path), repository) is expected
|
|
1305
|
+
|
|
1306
|
+
|
|
1307
|
+
def test_checkout_should_belong_to_the_target_repository_when_a_fork_checkout_names_it_as_upstream(
|
|
1308
|
+
tmp_path: Path,
|
|
1309
|
+
) -> None:
|
|
1310
|
+
subprocess.run(["git", "init", "-q", "-b", "main", str(tmp_path)], check=True)
|
|
1311
|
+
for each_name, each_url in (("origin", "https://github.com/me/r.git"), ("upstream", "https://github.com/o/r.git")):
|
|
1312
|
+
subprocess.run(["git", "-C", str(tmp_path), "remote", "add", each_name, each_url], check=True)
|
|
1313
|
+
assert gate.checkout_is_for(str(tmp_path), "o/r") is True
|
|
1314
|
+
assert gate.checkout_is_for(str(tmp_path), "x/r") is False
|
|
1315
|
+
|
|
1316
|
+
|
|
1317
|
+
def test_unproven_line_should_stay_denied_when_a_matching_checkout_lists_no_added_workflow(
|
|
1318
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
1319
|
+
) -> None:
|
|
1320
|
+
stale_checkout = _repository_with_change(tmp_path, "tool.py")
|
|
1321
|
+
subprocess.run(
|
|
1322
|
+
["git", "-C", str(stale_checkout), "remote", "set-url", "origin", "https://github.com/o/r.git"], check=True
|
|
1323
|
+
)
|
|
1324
|
+
added_path = f"{WORKFLOW_DIRECTORY}/{NEW_WORKFLOW_NAME}"
|
|
1325
|
+
compare_endpoint = "repos/o/r/compare/main...feature"
|
|
1326
|
+
monkeypatch.setattr(
|
|
1327
|
+
gate, "_gh_api_lines", _fake_gh_api({compare_endpoint: _workflow_compare_lines(2, [added_path])}, [])
|
|
1328
|
+
)
|
|
1329
|
+
payload = {
|
|
1330
|
+
"tool_name": "mcp__github__create_pull_request",
|
|
1331
|
+
"tool_input": {
|
|
1332
|
+
"owner": "o",
|
|
1333
|
+
"repo": "r",
|
|
1334
|
+
"base": "main",
|
|
1335
|
+
"head": "feature",
|
|
1336
|
+
"title": "t",
|
|
1337
|
+
"body": _workflow_body(AFTER_MERGE_SECTION),
|
|
1338
|
+
},
|
|
1339
|
+
"cwd": str(stale_checkout),
|
|
1340
|
+
}
|
|
1341
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=UNPROVEN_RUN_LINE)
|
|
1342
|
+
|
|
1343
|
+
|
|
1344
|
+
def test_unproven_line_should_stay_denied_when_a_longer_file_name_contains_the_workflow_name(tmp_path: Path) -> None:
|
|
1345
|
+
clone_path = _repository_with_workflow_change(tmp_path, "ci2.yml")
|
|
1346
|
+
unrelated_line = "Untested: the ci2.yml.bak migration path."
|
|
1347
|
+
after_merge_section = AFTER_MERGE_SECTION.replace(NEW_WORKFLOW_NAME, "ci2.yml")
|
|
1348
|
+
body = _workflow_body(after_merge_section).replace(UNPROVEN_RUN_LINE, unrelated_line)
|
|
1349
|
+
payload = _mcp_create_on_branch(body, clone_path)
|
|
1350
|
+
assert gate.missing_proof_reason(payload) == UNPROVEN_PROOF_REASON.format(admission=unrelated_line)
|
|
1351
|
+
|
|
1352
|
+
|
|
843
1353
|
@pytest.mark.parametrize(
|
|
844
1354
|
"admission",
|
|
845
1355
|
[
|
|
@@ -84,7 +84,25 @@ UNPROVEN_PROOF_REASON = (
|
|
|
84
84
|
" Do not open the pull request, or ask for review of an open one, until that output"
|
|
85
85
|
" is there. When it is a condition the change does not"
|
|
86
86
|
" target, move the line under a 'Not covered' heading at the same level as the proof"
|
|
87
|
-
" heading.
|
|
87
|
+
" heading. When the pull request adds a workflow file the default branch lacks, its first"
|
|
88
|
+
" run waits for the merge: add an 'After merge' heading that names the workflow file in"
|
|
89
|
+
" backticks and says who records the run link. Read " + PROOF_GUIDE_PATH + " for the full rule."
|
|
90
|
+
)
|
|
91
|
+
AFTER_MERGE_HEADING_PATTERN = r"^(#{1,6})[ \t]+after merge[ \t]*#*[ \t]*$"
|
|
92
|
+
AFTER_MERGE_RECORDER_PATTERN = r"\brecord(?:s|ed)?\b[^.;\n]{0,30}\b(?:run|link|url|result)s?\b"
|
|
93
|
+
AFTER_MERGE_NO_RECORDER_PATTERN = (
|
|
94
|
+
r"\b(?:nobody|no[ -]?one|not|never|cannot|can't|won't|doesn't)\b[^.;,\n]{0,60}\brecord(?:s|ed)?\b"
|
|
95
|
+
r"|\brecord(?:s|ed)?\s+(?:nothing|none|no)\b|\bno\s+records?\b"
|
|
96
|
+
)
|
|
97
|
+
AFTER_MERGE_SENTENCE_SEPARATOR_PATTERN = r"[;\n]|\.(?=\s|$)"
|
|
98
|
+
WORKFLOW_SUBJECT_PATTERN = r"\b(?:github actions|workflow)\b"
|
|
99
|
+
RUN_WORD_PATTERN = r"\bruns?\b"
|
|
100
|
+
GIT_URL_SUFFIX = ".git"
|
|
101
|
+
REMOTE_LINE_FIELD_COUNT = 2
|
|
102
|
+
WORKFLOW_NAME_MATCH_TEMPLATE = r"(?<![\w.-]){workflow_name}(?![\w-]|\.\w)"
|
|
103
|
+
NEW_WORKFLOW_PATH_PATTERN = r"^\.github/workflows/[^/]+\.ya?ml$"
|
|
104
|
+
ADDED_FILE_NAMES_QUERY = (
|
|
105
|
+
'(.files | length | tostring), (.files[] | select(.status == "added" or .status == "renamed" or .status == "copied") | .filename)'
|
|
88
106
|
)
|
|
89
107
|
DEFERRED_PROOF_REASON = (
|
|
90
108
|
"This pull request's 'Proof in practice' section mentions a merge: \"{deferral}\". Run that"
|
package/package.json
CHANGED
|
@@ -64,6 +64,7 @@ if _hooks_directory not in sys.path:
|
|
|
64
64
|
sys.path.insert(0, _hooks_directory)
|
|
65
65
|
|
|
66
66
|
from blocking.pull_request_proof import (
|
|
67
|
+
checkout_is_for,
|
|
67
68
|
missing_eval_section_reason,
|
|
68
69
|
missing_existing_work_reason,
|
|
69
70
|
missing_proof_reason,
|
|
@@ -159,12 +160,32 @@ def _description_heading_findings(body_text: str) -> list[DurablePostFinding]:
|
|
|
159
160
|
]
|
|
160
161
|
|
|
161
162
|
|
|
163
|
+
def _checkout_directory(head_branch: str | None, repository: str | None) -> str:
|
|
164
|
+
current_directory = str(Path.cwd())
|
|
165
|
+
if head_branch and repository and checkout_is_for(current_directory, repository):
|
|
166
|
+
return current_directory
|
|
167
|
+
return ""
|
|
168
|
+
|
|
169
|
+
|
|
162
170
|
def _pull_request_proof_findings(
|
|
163
|
-
title: str | None,
|
|
171
|
+
title: str | None,
|
|
172
|
+
body_text: str,
|
|
173
|
+
head_branch: str | None = None,
|
|
174
|
+
repository: str | None = None,
|
|
164
175
|
) -> list[DurablePostFinding]:
|
|
176
|
+
owner, _separator, repository_name = (repository or "").partition(
|
|
177
|
+
REPOSITORY_OWNER_SEPARATOR
|
|
178
|
+
)
|
|
165
179
|
create_payload = {
|
|
166
180
|
"tool_name": PROOF_GATE_CREATE_TOOL_NAME,
|
|
167
|
-
"tool_input": {
|
|
181
|
+
"tool_input": {
|
|
182
|
+
"title": title,
|
|
183
|
+
"body": body_text,
|
|
184
|
+
"head": head_branch,
|
|
185
|
+
"owner": owner,
|
|
186
|
+
"repo": repository_name,
|
|
187
|
+
},
|
|
188
|
+
"cwd": _checkout_directory(head_branch, repository),
|
|
168
189
|
}
|
|
169
190
|
all_reasons = (
|
|
170
191
|
missing_proof_reason(create_payload),
|
|
@@ -251,6 +272,7 @@ def _body_findings(
|
|
|
251
272
|
body_text: str | None,
|
|
252
273
|
head_branch: str | None = None,
|
|
253
274
|
title: str | None = None,
|
|
275
|
+
repository: str | None = None,
|
|
254
276
|
) -> list[DurablePostFinding]:
|
|
255
277
|
if body_text is None:
|
|
256
278
|
return []
|
|
@@ -267,7 +289,9 @@ def _body_findings(
|
|
|
267
289
|
else:
|
|
268
290
|
if action == ACTION_PR_CREATE:
|
|
269
291
|
all_findings.extend(_description_heading_findings(body_text))
|
|
270
|
-
all_findings.extend(
|
|
292
|
+
all_findings.extend(
|
|
293
|
+
_pull_request_proof_findings(title, body_text, head_branch, repository)
|
|
294
|
+
)
|
|
271
295
|
if action == ACTION_GITHUB_MCP_COMMENT:
|
|
272
296
|
all_findings.extend(_comment_length_findings(body_text))
|
|
273
297
|
all_findings.extend(_contrast_framing_findings(body_text))
|
|
@@ -307,7 +331,7 @@ def lint_durable_post(
|
|
|
307
331
|
return (
|
|
308
332
|
*_title_findings(title),
|
|
309
333
|
*_private_term_findings(title, repository),
|
|
310
|
-
*_body_findings(action, body_text, head_branch, title),
|
|
334
|
+
*_body_findings(action, body_text, head_branch, title, repository),
|
|
311
335
|
*_private_term_findings(body_text, repository),
|
|
312
336
|
)
|
|
313
337
|
|
|
@@ -769,3 +769,138 @@ def test_cli_accepts_a_repository(tmp_path: Path) -> None:
|
|
|
769
769
|
)
|
|
770
770
|
assert completed_process.returncode == 0
|
|
771
771
|
assert completed_process.stderr == ""
|
|
772
|
+
|
|
773
|
+
|
|
774
|
+
def _repository_adding_workflow(tmp_path: Path, workflow_name: str) -> Path:
|
|
775
|
+
git_identity = ["-c", "user.name=t", "-c", "user.email=t@example.com"]
|
|
776
|
+
repository_path = tmp_path / "repository"
|
|
777
|
+
subprocess.run(
|
|
778
|
+
["git", "init", "-q", "-b", "main", str(repository_path)], check=True
|
|
779
|
+
)
|
|
780
|
+
(repository_path / "readme.md").write_text("base\n", encoding="utf-8")
|
|
781
|
+
subprocess.run(["git", "-C", str(repository_path), "add", "."], check=True)
|
|
782
|
+
subprocess.run(
|
|
783
|
+
[
|
|
784
|
+
"git",
|
|
785
|
+
"-C",
|
|
786
|
+
str(repository_path),
|
|
787
|
+
*git_identity,
|
|
788
|
+
"commit",
|
|
789
|
+
"-q",
|
|
790
|
+
"-m",
|
|
791
|
+
"base",
|
|
792
|
+
],
|
|
793
|
+
check=True,
|
|
794
|
+
)
|
|
795
|
+
subprocess.run(
|
|
796
|
+
["git", "-C", str(repository_path), "checkout", "-q", "-b", "feature"],
|
|
797
|
+
check=True,
|
|
798
|
+
)
|
|
799
|
+
subprocess.run(
|
|
800
|
+
["git", "-C", str(repository_path), "remote", "add", "origin", "https://github.com/o/r.git"],
|
|
801
|
+
check=True,
|
|
802
|
+
)
|
|
803
|
+
workflow_directory = repository_path / ".github" / "workflows"
|
|
804
|
+
workflow_directory.mkdir(parents=True)
|
|
805
|
+
(workflow_directory / workflow_name).write_text("name: added\n", encoding="utf-8")
|
|
806
|
+
subprocess.run(["git", "-C", str(repository_path), "add", "."], check=True)
|
|
807
|
+
subprocess.run(
|
|
808
|
+
[
|
|
809
|
+
"git",
|
|
810
|
+
"-C",
|
|
811
|
+
str(repository_path),
|
|
812
|
+
*git_identity,
|
|
813
|
+
"commit",
|
|
814
|
+
"-q",
|
|
815
|
+
"-m",
|
|
816
|
+
"change",
|
|
817
|
+
],
|
|
818
|
+
check=True,
|
|
819
|
+
)
|
|
820
|
+
return repository_path
|
|
821
|
+
|
|
822
|
+
|
|
823
|
+
def _unproven_workflow_body(after_merge_section: str) -> str:
|
|
824
|
+
return (
|
|
825
|
+
VALID_PR_BODY.replace(
|
|
826
|
+
"on the new body.",
|
|
827
|
+
"on the new body.\n\nNot proven here: the live GitHub Actions run.",
|
|
828
|
+
)
|
|
829
|
+
+ after_merge_section
|
|
830
|
+
)
|
|
831
|
+
|
|
832
|
+
|
|
833
|
+
def should_accept_an_unproven_run_when_the_branch_adds_the_workflow_its_after_merge_section_names(
|
|
834
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
835
|
+
) -> None:
|
|
836
|
+
monkeypatch.chdir(_repository_adding_workflow(tmp_path, "release-hold.yml"))
|
|
837
|
+
after_merge_section = (
|
|
838
|
+
"\n## After merge\n\nThe first owner approval runs `release-hold.yml`."
|
|
839
|
+
" The agent records that run link here.\n"
|
|
840
|
+
)
|
|
841
|
+
all_findings = durable_post_lint.lint_durable_post(
|
|
842
|
+
action="pr-create",
|
|
843
|
+
title="feat: release the hold label",
|
|
844
|
+
body_text=_unproven_workflow_body(after_merge_section),
|
|
845
|
+
head_branch="feature",
|
|
846
|
+
repository="o/r",
|
|
847
|
+
)
|
|
848
|
+
assert all_findings == ()
|
|
849
|
+
|
|
850
|
+
|
|
851
|
+
def should_report_an_unproven_run_when_the_checkout_belongs_to_another_repository(
|
|
852
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
853
|
+
) -> None:
|
|
854
|
+
monkeypatch.chdir(_repository_adding_workflow(tmp_path, "release-hold.yml"))
|
|
855
|
+
after_merge_section = (
|
|
856
|
+
"\n## After merge\n\nThe first owner approval runs `release-hold.yml`."
|
|
857
|
+
" The agent records that run link here.\n"
|
|
858
|
+
)
|
|
859
|
+
all_findings = durable_post_lint.lint_durable_post(
|
|
860
|
+
action="pr-create",
|
|
861
|
+
title="feat: release the hold label",
|
|
862
|
+
body_text=_unproven_workflow_body(after_merge_section),
|
|
863
|
+
head_branch="feature",
|
|
864
|
+
repository="x/y",
|
|
865
|
+
)
|
|
866
|
+
assert [each_finding.code for each_finding in all_findings] == [
|
|
867
|
+
durable_post_lint.PULL_REQUEST_PROOF_FINDING_CODE
|
|
868
|
+
]
|
|
869
|
+
|
|
870
|
+
|
|
871
|
+
def should_report_an_unproven_run_when_no_head_branch_is_named(
|
|
872
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
873
|
+
) -> None:
|
|
874
|
+
monkeypatch.chdir(_repository_adding_workflow(tmp_path, "release-hold.yml"))
|
|
875
|
+
after_merge_section = (
|
|
876
|
+
"\n## After merge\n\nThe first owner approval runs `release-hold.yml`."
|
|
877
|
+
" The agent records that run link here.\n"
|
|
878
|
+
)
|
|
879
|
+
all_findings = durable_post_lint.lint_durable_post(
|
|
880
|
+
action="pr-create",
|
|
881
|
+
title="feat: release the hold label",
|
|
882
|
+
body_text=_unproven_workflow_body(after_merge_section),
|
|
883
|
+
head_branch=None,
|
|
884
|
+
repository="o/r",
|
|
885
|
+
)
|
|
886
|
+
assert [each_finding.code for each_finding in all_findings] == [
|
|
887
|
+
durable_post_lint.PULL_REQUEST_PROOF_FINDING_CODE
|
|
888
|
+
]
|
|
889
|
+
|
|
890
|
+
|
|
891
|
+
def should_report_an_unproven_run_when_the_after_merge_section_names_another_workflow(
|
|
892
|
+
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
893
|
+
) -> None:
|
|
894
|
+
monkeypatch.chdir(_repository_adding_workflow(tmp_path, "release-hold.yml"))
|
|
895
|
+
after_merge_section = "\n## After merge\n\nThe first owner approval runs `other.yml`. The agent records that run link here.\n"
|
|
896
|
+
all_findings = durable_post_lint.lint_durable_post(
|
|
897
|
+
action="pr-create",
|
|
898
|
+
title="feat: release the hold label",
|
|
899
|
+
body_text=_unproven_workflow_body(after_merge_section),
|
|
900
|
+
head_branch="feature",
|
|
901
|
+
repository="o/r",
|
|
902
|
+
)
|
|
903
|
+
assert [each_finding.code for each_finding in all_findings] == [
|
|
904
|
+
durable_post_lint.PULL_REQUEST_PROOF_FINDING_CODE
|
|
905
|
+
]
|
|
906
|
+
assert "Not proven here: the live GitHub Actions run." in all_findings[0].message
|